Categorical features with Linear Regression

Hi,

While training a dataset from kaggle i realized that i can also use categorical features for my linear regression model. Then i tried fitrlinear() function to use regularization but it didn't work because of the categorical features. This made me think that i can add categorical features with normal linear regression but the algorithm doesn't use them, just goes with numerical ones, and it sorta makes sense because there isn't any leastsquare equation for categorical values yet when i omit categorical values for my linear regression model i saw that it makes a lot of difference.

How does linear regression uses categorical variables and can i use regularization with a table that has categorical features ?