From float to string in random forest fitting

AI & Machine learning
1

i have this error but i don't know what is necessary…i have check all the code and seems ok…also i have check the row and columns and also nan date in the data but everything is ok. so thank you for the help. 

 

alueError                                Traceback (most recent call last)
Input In [57], in <cell line: 5>()
      1 rf_model = RandomForestClassifier(
      2     n_estimators=3, max_features=3, max_depth=2, random_state=4)
      4 # Fitting del modello sui dati
----> 5 rf_model.fit(X_train, y_train['signal'])

File ~\anaconda3\lib\site-packages\sklearn\ensemble\_forest.py:327, in BaseForest.fit(self, X, y, sample_weight)
    325 if issparse(y):
    326     raise ValueError("sparse multilabel-indicator for y is not supported.")
--> 327 X, y = self._validate_data(
    328     X, y, multi_output=True, accept_sparse="csc", dtype=DTYPE
    329 )
    330 if sample_weight is not None:
    331     sample_weight = _check_sample_weight(sample_weight, X)

File ~\anaconda3\lib\site-packages\sklearn\base.py:581, in BaseEstimator._validate_data(self, X, y, reset, validate_separately, **check_params)
    579         y = check_array(y, **check_y_params)
    580     else:
--> 581         X, y = check_X_y(X, y, **check_params)
    582     out = X, y
    584 if not no_val_X and check_params.get("ensure_2d", True):

File ~\anaconda3\lib\site-packages\sklearn\utils\validation.py:964, in check_X_y(X, y, accept_sparse, accept_large_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, multi_output, ensure_min_samples, ensure_min_features, y_numeric, estimator)
    961 if y is None:
    962     raise ValueError("y cannot be None")
--> 964 X = check_array(
    965     X,
    966     accept_sparse=accept_sparse,
    967     accept_large_sparse=accept_large_sparse,
    968     dtype=dtype,
    969     order=order,
    970     copy=copy,
    971     force_all_finite=force_all_finite,
    972     ensure_2d=ensure_2d,
    973     allow_nd=allow_nd,
    974     ensure_min_samples=ensure_min_samples,
    975     ensure_min_features=ensure_min_features,
    976     estimator=estimator,
    977 )
    979 y = _check_y(y, multi_output=multi_output, y_numeric=y_numeric)
    981 check_consistent_length(X, y)

File ~\anaconda3\lib\site-packages\sklearn\utils\validation.py:746, in check_array(array, accept_sparse, accept_large_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, ensure_min_samples, ensure_min_features, estimator)
    744         array = array.astype(dtype, casting="unsafe", copy=False)
    745     else:
--> 746         array = np.asarray(array, order=order, dtype=dtype)
    747 except ComplexWarning as complex_warning:
    748     raise ValueError(
    749         "Complex data not supported\n{}\n".format(array)
    750     ) from complex_warning

File ~\anaconda3\lib\site-packages\pandas\core\generic.py:2064, in NDFrame.__array__(self, dtype)
   2063 def __array__(self, dtype: npt.DTypeLike | None = None) -> np.ndarray:
-> 2064     return np.asarray(self._values, dtype=dtype)

ValueError: could not convert string to float: '2022-06-13 10:22:00-04:00'


​
2

Hi Irene,



One possible reason for this error can be the incorrect parsing of DateTime as a column rather than an index. You can try parsing the DateTime as an index. If the error still persists please feel free to share the code.



Hope this helps!



Thanks,

Akshay

 

3

Hi Choudhary, in the data analisys i have reliable the data in the on DateTime and verify also the column and drop the double information…but i have always the same error…probably the error is in X_train use in the random forest. but i don t know exactly…maybe could you help with an other advice? thank you

4

Hi Irene,



Can you please share the code so we can thoroughly look at the same for debugging it?



Thanks,

Akshay

5

hi akshay, i have two problem than before.

one is in the check of stationarity

Check for stationarity

for col in X.columns:

    if stationary(nasdaq_minute_data[col]) == 'not stationary':

        print('%s is not stationary. Dropping it.' % col)

        X.drop(columns=[col], axis=1, inplace=True)

    else:

        print('%s is stationary.' % col)   

the error is:

MissingDataError: exog contains inf or nans


and the other error in the same code and after the errore above is:

single_day_prediction=rf_model.predict(unseen_data_single_day)

NotFittedError: 
This RandomForestClassifier instance is not fitted yet. 
Call 'fit' with appropriate arguments before using this estimator.

thank you for the help.
6

Hi Irene,



The first error is because of some missing values in the dataset. You need to pre-process the data before checking the stationarity. You can also refer to the below link to learn more about handling missing values - 



https://pandas.pydata.org/pandas-docs/stable/user_guide/missing_data.html



The second error is because the random forest model is not fitted. First, you must fit the model on the training dataset and then call the predict function. You can refer to the following example for the same - 



https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html



Hope this helps!



Thanks,

Akshay

7

Hi akshay, thank you for you disponibility.

I have still now the problem…i have check from the beginning my raw data…so there is not any nan value, and the index is on datetime schedule, and before the stationary and correlation check i have drop the data does not correct.

i have also check in the x train y train ecc…the number of row and everything is ok… i don t understand why i cannot fit my model.

there is a problem with my data i think …

i am using the nasdaq index minute data import from yahoo finance.

i have right now the errore after the code string of fit model

you cannot convert…

ValueError: could not convert string to float: '2022-07-07 10:22:00-04:00'

hope in your help. 

belove you will find some parts of my code. 

#IMPORT DATA FROM YAHOO FINANCE

nasdaq_minute_data=yf.download(tickers="^IXIC",period="5d",interval="1m",index_col=0)
nasdaq_minute_data.index=pd.to_datetime(nasdaq_minute_data.index)
nasdaq_minute_data.head()

BELOVE AFTER STATIONARY, AFTER TRAIN TEST SPLIT, CORRELATION..

I HAVE DO THAT WITH EVERY DATA TRAIN AND TEST..AND ITS OK(THE SHAPE ARE CORRECT)
FOR EXAMPLE WITH Y_TEST

y_test.isna().sum()
print('number of rows:',y_test.shape[0])#stampa delle righe totali
number of rows: 364
8

Hi Irene,



The error is because you cannot directly pass str to your model fit() method. You can refer to the following documentation -



https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html#sklearn.ensemble.RandomForestClassifier.fit 



You can also refer to the following link for a detailed explanation about the same - 



https://datascience.stackexchange.com/questions/5226/strings-as-features-in-decision-tree-random-forest



Hope this helps!



Thanks,

Akshay

9

hi i still have the same problem…

could not convert string to float…i think because the date is in the format using :  and -

could not convert string to float: '2022-06-24 09:30:00-04:00'

(i have used the datetime index)

maybe do you have a solution for drop the : and - from the datetime?
 
10

Hi Irene,



Can you please share a minimal part of the code required to reproduce the issue so that we can have a look at the same and provide a solution?



Thanks,

Akshay