XGBoost: OS Error : [WinError -529697949] Windows Error 0xe06d7363 running XGBClassifier with large dataset, CPU Mode

1

Getting this error while trying to run XGBClassifier and GridsearchCV for hyperparameter optimization. I have seen this issue being opened in Github but closed and marked resolved but no solution provided. Has anyone actually found a soultion to this error?

My dataset:

X = np array with 350000 rows and 1715 columns (after one hot encoding)
y = 350000 rows and 1 column (target) 

My Code:

X = train.drop(['Breakage'], axis=1,)  #features (read from dataframe)
y = train['Breakage']    #target (read from dataframe)

X= X.as_matrix()    #convert to np array
y= y.as_matrix()    #convert to np array
y = np.reshape(y,(-1, 1))    #reshape array

X = X.astype('uint8')  #Change dtype to avoid overcommmit error in windows
y = y.astype('uint8')  #Change dtype to avoid overcommmit error in windows

#define estimators and learning rate
model = XGBClassifier()
n_estimators = [100, 200, 300, 400, 500]
learning_rate = [0.0001, 0.001, 0.01, 0.1]

# GRidSearchCV
param_grid = dict(learning_rate=learning_rate, n_estimators=n_estimators)
kfold = StratifiedKFold(n_splits=5, shuffle=True, random_state=7)
grid_search = GridSearchCV(model, param_grid, n_jobs=-1, cv=kfold)
grid_result = grid_search.fit(X,y)

The output Error:

OSError: [WinError -529697949] Windows Error 0xe06d7363

Can anyone please tell me what i am doing wrong

python-3.x
xgboost
asked on Stack Overflow Jan 28, 2020 by Shameendra

2 Answers

0

I have same problem as yours. This answer will help you. Windows Error using XGBoost with python Change the 'xgb.fit()' sklearn api to 'xgb.train()' will solve this problem.

answered on Stack Overflow Jul 2, 2020 by yanzhen xu
0

I had this issue and it was something to do with the installation of xgboost.

I was using code that already worked before, and although there was a lot of memory usage, it is not a memory error.

What worked for me was

  1. Close the IDE
  2. Uninstall xgboost
  3. Upgrade pip (not sure if necessary)
  4. Reinstall xgboost
  5. Rebuild and install python project
  6. Run again

Source of inspiration from this question here: Windows Error 0xe06d7363 when using Cross Validation XGboost

answered on Stack Overflow Jan 27, 2021 by adelzel

User contributions licensed under CC BY-SA 3.0