🏅 Starting Info

💡 Evaluation metrics for Classification models:

• Accuracy: Accuracy is the most straightforward metric and measures the overall correctness of the model's predictions. It calculates the ratio of correct predictions to the total number of predictions made.

• Precision: Precision measures the proportion of correctly predicted positive instances (true positives) out of the total instances predicted as positive (true positives + false positives). It indicates the model's ability to avoid false positives.

• Recall (Sensitivity or True Positive Rate): Recall calculates the proportion of correctly predicted positive instances (true positives) out of the total actual positive instances (true positives + false negatives). It indicates the model's ability to identify positive instances.

• Specificity (True Negative Rate): Specificity measures the proportion of correctly predicted negative instances (true negatives) out of the total actual negative instances (true negatives + false positives). It indicates the model's ability to identify negative instances.

• F1 Score: The F1 score is the harmonic mean of precision and recall. It provides a balanced measure between precision and recall, considering both false positives and false negatives. It is useful when the class distribution is imbalanced.

• Area Under the ROC Curve (AUC-ROC): The ROC curve represents the trade-off between true positive rate (sensitivity) and false positive rate (1 - specificity) at various classification thresholds. AUC-ROC measures the overall performance of the model across different thresholds, providing a single value to compare different models. A higher AUC-ROC indicates better model performance.

• Cohen's Kappa: Cohen's Kappa is a statistic that measures the agreement between the predicted and actual classifications, while taking into account the agreement that could occur by chance. It is useful when evaluating models with imbalanced class distributions.

</font>

💡 Popular Classification Models:

• Logistic Regression: Logistic regression models the probability of a binary or multi-class outcome using a logistic function. It is widely used for binary classification tasks and can be extended to handle multiple classes.

  from sklearn.linear_model import LogisticRegression
  

• Decision Tree: Decision tree classifiers partition the feature space into regions based on feature values. Each partition represents a leaf node with a predicted class label.

  from sklearn.tree import DecisionTreeClassifier
  

• Random Forest: Random forest classifiers are ensemble models that combine multiple decision trees. They improve prediction accuracy by averaging the predictions of individual trees.

  from sklearn.ensemble import RandomForestClassifier
  

• Gradient Boosting: Gradient boosting classifiers build an ensemble of weak learners, typically decision trees, in a sequential manner. Each subsequent model corrects the errors made by the previous models, leading to improved predictions.

  from sklearn.ensemble import GradientBoostingClassifier
  

• Support Vector Machines (SVM): SVM classifiers find the optimal hyperplane that separates the data into different classes with the maximum margin. They can handle linear and nonlinear classification tasks.

  from sklearn.svm import SVC
  

• K-Nearest Neighbors (KNN): KNN classifiers predict the class label of an instance by considering the labels of its k nearest neighbors in the feature space.

  from sklearn.neighbors import KNeighborsClassifier
  

• Naive Bayes: Naive Bayes classifiers are probabilistic models that use Bayes' theorem with strong independence assumptions between features. They are fast and effective for text classification and other tasks.

  from sklearn.naive_bayes import GaussianNB
  

• Neural Networks: Neural network classifiers utilize deep learning architectures to model complex relationships between predictors and class labels. They consist of multiple interconnected layers of artificial neurons that learn to approximate the target function.

  from sklearn.neural_network import MLPClassifier
  

• AdaBoost: AdaBoost is an ensemble method that combines multiple weak classifiers to create a strong classifier. It assigns higher weights to misclassified instances, focusing on difficult examples.

  from sklearn.ensemble import AdaBoostClassifier
  

• Gradient Boosting with XGBoost: XGBoost is an optimized gradient boosting classifier known for its high performance and scalability. It offers various boosting algorithms and regularization techniques.

  import xgboost as xgb
  

• Gradient Boosting with LightGBM: LightGBM is another gradient boosting classifier that provides high efficiency and handles large-scale data. It offers faster training speed and supports categorical features.

  import lightgbm as lgb
  

• Gradient Boosting with CatBoost: CatBoost is a gradient boosting classifier that supports categorical features and incorporates innovative techniques such as ordered boosting and symmetric trees.

  import catboost as cb
  

• Logistic Regression with L1 Regularization (Lasso): L1 regularized logistic regression can perform feature selection by setting some coefficients to exactly zero. It helps to reduce the number of irrelevant features.

  from sklearn.linear_model import LogisticRegression
  

• Support Vector Machines with RBF Kernel: SVM classifiers with radial basis function (RBF) kernel can handle nonlinear classification tasks by mapping the input space into a higher-dimensional feature space.

  from sklearn.svm import SVC
  

• Extra Trees: Extra Trees classifiers are ensemble models similar to random forests but with some differences in the way they build individual trees and handle feature splits.

  from sklearn.ensemble import ExtraTreesClassifier
  

• Gaussian Process: Gaussian process classifiers model the decision boundary as a distribution over functions. They provide a nonparametric approach to classification that captures uncertainty in predictions.

  from sklearn.gaussian_process import GaussianProcessClassifier
  

• Nearest Centroid: Nearest Centroid classifiers compute the centroids of each class and classify instances based on the nearest centroid. They are simple and fast but assume that all features are equally important.

  from sklearn.neighbors import NearestCentroid
  

• Quadratic Discriminant Analysis (QDA): QDA classifiers model the probability distribution of each class using a quadratic function. They assume a quadratic decision boundary and can capture complex relationships.

  from sklearn.discriminant_analysis import QuadraticDiscriminantAnalysis
  

• Linear Discriminant Analysis (LDA): LDA classifiers find a linear combination of features that maximizes the separation between classes. They project the data onto a lower-dimensional space while preserving class information.

  from sklearn.discriminant_analysis import LinearDiscriminantAnalysis
  

• HistGradientBoosting Classifier: HistGradientBoosting is a histogram-based gradient boosting classifier that provides fast and accurate predictions. It can handle large-scale datasets.

  from sklearn.ensemble import HistGradientBoostingClassifier
  

• Bagging: Bagging classifiers combine multiple base estimators, such as decision trees or SVMs, by training them on random subsets of the training data. They reduce variance and improve generalization.

  from sklearn.ensemble import BaggingClassifier
  

• Calibrated Classifier: Calibrated classifiers assign class probabilities calibrated to the predicted scores. They can improve the reliability of probabilistic predictions.

  from sklearn.calibration import CalibratedClassifierCV
  

• Multi-layer Perceptron (MLP) Classifier: MLP classifiers are feedforward neural networks with multiple hidden layers. They can learn complex patterns and are effective for a wide range of classification tasks.

  from sklearn.neural_network import MLPClassifier
  

</font>

🛫 Imports

'''
--------------------------------------------------------
CLASSIFICATION MODELS
--------------------------------------------------------
'''
from sklearn.ensemble import RandomForestClassifier
from sklearn.ensemble import BaggingClassifier
from sklearn.linear_model import LogisticRegression
from sklearn import svm
from sklearn.naive_bayes import GaussianNB
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.ensemble import AdaBoostClassifier
from xgboost import XGBClassifier
from lightgbm import LGBMClassifier
from catboost import CatBoostClassifier
from sklearn.neighbors import KNeighborsClassifier
from sklearn.ensemble import VotingClassifier
from sklearn.svm import SVC

'''
--------------------------------------------------------
FEAUTURE ENGINEERING
--------------------------------------------------------
'''

from sklearn.base import BaseEstimator, TransformerMixin
from sklearn.metrics import plot_confusion_matrix, classification_report
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import MinMaxScaler
from sklearn.model_selection import GridSearchCV

'''
--------------------------------------------------------
OTHER
--------------------------------------------------------
'''
import missingno as msno
import tensorflow as tf
from tensorflow import keras
import numpy as np
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.metrics import mean_squared_error, r2_score, accuracy_score, roc_auc_score
from sklearn.metrics import accuracy_score, r2_score, roc_auc_score, precision_score, recall_score, f1_score
import seaborn as sns
import plotly.express as px
import pandas as pd
import catboost as cb
from IPython.display import Image
import warnings
import optuna

def ignoreWarnings():
    warnings.filterwarnings('ignore')

💡 The first step is to import the data

data = pd.read_csv('Csv Files/heart.csv')
pd.set_option('display.max_columns', None) #This will display all the columns in the dataframe
data

	Age	Sex	ChestPainType	RestingBP	Cholesterol	FastingBS	RestingECG	MaxHR	ExerciseAngina	Oldpeak	ST_Slope	HeartDisease
0	40	M	ATA	140	289	0	Normal	172	N	0.0	Up	0
1	49	F	NAP	160	180	0	Normal	156	N	1.0	Flat	1
2	37	M	ATA	130	283	0	ST	98	N	0.0	Up	0
3	48	F	ASY	138	214	0	Normal	108	Y	1.5	Flat	1
4	54	M	NAP	150	195	0	Normal	122	N	0.0	Up	0
...	...	...	...	...	...	...	...	...	...	...	...	...
913	45	M	TA	110	264	0	Normal	132	N	1.2	Flat	1
914	68	M	ASY	144	193	1	Normal	141	N	3.4	Flat	1
915	57	M	ASY	130	131	0	Normal	115	Y	1.2	Flat	1
916	57	F	ATA	130	236	0	LVH	174	N	0.0	Flat	1
917	38	M	NAP	138	175	0	Normal	173	N	0.0	Up	0

918 rows × 12 columns

📊 EDA

Variable Name	Units or Categories	Description	Qaul / Quan	Type
Age	Years	How many years the person has been alive	Quantitative	Discrete
Sex	Male (M); Female(F)	What gender the person is	Qaulitative	Nominal
ChestPainType	Typical Angina (TA); Atypical Angina (ATA); Non-Anginal Pain (NAP); Asymptomatic (ASY)	What type of chest pain is the person experiencing	Qualitative	Nomial
RestingBP	mm Hg	The Patients resting Blood pressure	Quantitative	Discrete
Cholesterol	mm/dl	Patients Cholesterol levels	Quantitative	Discrete
FastingBS	1: if FastingBS > 120 mg/dl, 0: otherwise	Fasting blood sugar	Qualitative	Nominal
RestingECG	Normal (Normal); Having ST-T wave abnormality (ST); Showing probable or definite left ventricular hypertrophy by Estes' criteria (LVH)	The patients Resting ECG	Qualitative	Nominal
MaxHR	Between 60 and 202	The patients maximum heart rate achieved	Quantitative	Discrete
ExerciseAngina	Yes (Y); No (N)	If there was an exercise-induced angina	Qualitative	Nominal
Oldpeak	ST [Numeric value measured in depression]	ST depression induced by exercise relative to rest	Quantitative	Continuous
ST_Slope	Upsloping (Up); Flat (Flat); Downsloping (Down)	The slope of the peak exercise ST segment	Qualitative	Nominal
HeartDisease	Heart Disease (1); Normal (0)	Output class that describes if someone has heart disease or not	Qualitative	Nominal

def summary(df):
        # Print the shape of the DataFrame
        print(f'data shape: {df.shape}')
        # Create a summary DataFrame
        summ = pd.DataFrame(df.dtypes, columns=['data type'])
        # Calculate the number of missing values
        summ['#missing'] = df.isnull().sum().values
        # Calculate the percentage of missing values
        summ['%missing'] = df.isnull().sum().values / len(df) * 100
        # Calculate the number of unique values
        summ['#unique'] = df.nunique().values
        # Create a descriptive DataFrame
        desc = pd.DataFrame(df.describe(include='all').transpose())
        # Add the minimum, maximum, and first three values to the summary DataFrame
        summ['min'] = desc['min'].values
        summ['max'] = desc['max'].values
        summ['first value'] = df.loc[0].values
        summ['second value'] = df.loc[1].values
        summ['third value'] = df.loc[2].values

        # Return the summary DataFrame
        return summ
    
summary(data)

data shape: (918, 12)

	data type	#missing	%missing	#unique	min	max	first value	second value	third value
Age	int64	0	0.0	50	28.0	77.0	40	49	37
Sex	object	0	0.0	2	NaN	NaN	M	F	M
ChestPainType	object	0	0.0	4	NaN	NaN	ATA	NAP	ATA
RestingBP	int64	0	0.0	67	0.0	200.0	140	160	130
Cholesterol	int64	0	0.0	222	0.0	603.0	289	180	283
FastingBS	int64	0	0.0	2	0.0	1.0	0	0	0
RestingECG	object	0	0.0	3	NaN	NaN	Normal	Normal	ST
MaxHR	int64	0	0.0	119	60.0	202.0	172	156	98
ExerciseAngina	object	0	0.0	2	NaN	NaN	N	N	N
Oldpeak	float64	0	0.0	53	-2.6	6.2	0.0	1.0	0.0
ST_Slope	object	0	0.0	3	NaN	NaN	Up	Flat	Up
HeartDisease	int64	0	0.0	2	0.0	1.0	0	1	0

for col in data.select_dtypes(include=[np.number]).columns:
  print(f"{col} has {data[col].nunique()} unique value")

Age has 50 unique value
RestingBP has 67 unique value
Cholesterol has 222 unique value
FastingBS has 2 unique value
MaxHR has 119 unique value
Oldpeak has 53 unique value
HeartDisease has 2 unique value

string_col = data.select_dtypes(include="object").columns
data[string_col] = data[string_col].astype("string")
data.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 918 entries, 0 to 917
Data columns (total 12 columns):
 #   Column          Non-Null Count  Dtype  
---  ------          --------------  -----  
 0   Age             918 non-null    int64  
 1   Sex             918 non-null    string 
 2   ChestPainType   918 non-null    string 
 3   RestingBP       918 non-null    int64  
 4   Cholesterol     918 non-null    int64  
 5   FastingBS       918 non-null    int64  
 6   RestingECG      918 non-null    string 
 7   MaxHR           918 non-null    int64  
 8   ExerciseAngina  918 non-null    string 
 9   Oldpeak         918 non-null    float64
 10  ST_Slope        918 non-null    string 
 11  HeartDisease    918 non-null    int64  
dtypes: float64(1), int64(6), string(5)
memory usage: 86.2 KB

💡 Let's reduce the size of the variables to help us save memory

int_col = data.select_dtypes(include="int64").columns
float_col = data.select_dtypes(include="float64").columns
#Convert to int8, int16, int32, float8, float16, float32 to save memory
for col in int_col:
    if data[col].max()<128: 
        data[col] = data[col].astype('int8')
    elif data[col].max()<32768: 
        data[col] = data[col].astype('int16')
    else: 
        data[col].astype('int32')
        
for col in float_col:   
    if data[col].max()<32768: 
        data[col] = data[col].astype('float16')
    else: 
        data[col].astype('float32')
        
data.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 918 entries, 0 to 917
Data columns (total 12 columns):
 #   Column          Non-Null Count  Dtype  
---  ------          --------------  -----  
 0   Age             918 non-null    int8   
 1   Sex             918 non-null    string 
 2   ChestPainType   918 non-null    string 
 3   RestingBP       918 non-null    int16  
 4   Cholesterol     918 non-null    int16  
 5   FastingBS       918 non-null    int8   
 6   RestingECG      918 non-null    string 
 7   MaxHR           918 non-null    int16  
 8   ExerciseAngina  918 non-null    string 
 9   Oldpeak         918 non-null    float16
 10  ST_Slope        918 non-null    string 
 11  HeartDisease    918 non-null    int8   
dtypes: float16(1), int16(3), int8(3), string(5)
memory usage: 45.8 KB

💡 Let's grab the column names for categorical columns and numerical columns

cat_cols = data.select_dtypes("string").columns.to_list()
cat_cols.append('FastingBS')
print(f'''The categorical columns are 
      {cat_cols}
      ''')



num_cols = [i for i in data.columns if i not in cat_cols and i != 'HeartDisease']
print(f'''The categorical columns are 
      {num_cols}
      ''')

The categorical columns are 
      ['Sex', 'ChestPainType', 'RestingECG', 'ExerciseAngina', 'ST_Slope', 'FastingBS']
      
The categorical columns are 
      ['Age', 'RestingBP', 'Cholesterol', 'MaxHR', 'Oldpeak']
      

data.describe()

	Age	RestingBP	Cholesterol	FastingBS	MaxHR	Oldpeak	HeartDisease
count	918.000000	918.000000	918.000000	918.000000	918.000000	918.000000	918.000000
mean	53.510893	132.396514	198.799564	0.233115	136.809368	0.887207	0.553377
std	9.432617	18.514154	109.384145	0.423046	25.460334	1.066406	0.497414
min	28.000000	0.000000	0.000000	0.000000	60.000000	-2.599609	0.000000
25%	47.000000	120.000000	173.250000	0.000000	120.000000	0.000000	0.000000
50%	54.000000	130.000000	223.000000	0.000000	138.000000	0.600098	1.000000
75%	60.000000	140.000000	267.000000	0.000000	156.000000	1.500000	1.000000
max	77.000000	200.000000	603.000000	1.000000	202.000000	6.199219	1.000000

plt.figure(figsize=(10, 8))
sns.scatterplot(data=data,x='Cholesterol',y='RestingBP',hue='RestingBP')

<AxesSubplot:xlabel='Cholesterol', ylabel='RestingBP'>

💡 This could be a bit of a problem.

There seems to be a few records with Cholesterol = 0 and 1 record with RestingBP = 0.
The chances of a living person having a Cholesterol or BP = 0 is probably too low to consider.
So these are likely input errors.
So we need to fix them somehow.

Let's convert the 0s to nulls and then visualise them </font>

data['Cholesterol'].replace(0,np.nan,inplace=True)
data['RestingBP'].replace(0,np.nan,inplace=True)

msno.matrix(data)

<AxesSubplot:>

💡 Findings

It looks like there are null values in the centre of the data.
Assuming these values where collected in chronological order, there was a time where someone was inputting values incorrectly.
Or one of the sources was unreliable.

When we build the model we can experiment with different techniques to fill these gaps.
For now we will drop the values which are = 0.
For the sake of analysing the data. </font>

data = data.dropna()
data.describe()

	Age	RestingBP	Cholesterol	FastingBS	MaxHR	Oldpeak	HeartDisease
count	746.000000	746.000000	746.000000	746.000000	746.000000	746.000000	746.000000
mean	52.882038	133.022788	244.635389	0.167560	140.226542	0.901367	0.477212
std	9.505888	17.282750	59.153524	0.373726	24.524107	1.072266	0.499816
min	28.000000	92.000000	85.000000	0.000000	69.000000	-0.099976	0.000000
25%	46.000000	120.000000	207.250000	0.000000	122.000000	0.000000	0.000000
50%	54.000000	130.000000	237.000000	0.000000	140.000000	0.500000	0.000000
75%	59.000000	140.000000	275.000000	0.000000	160.000000	1.500000	1.000000
max	77.000000	200.000000	603.000000	1.000000	202.000000	6.199219	1.000000

sns.scatterplot(data=data,x='Cholesterol',y='RestingBP',hue='RestingBP')

<AxesSubplot:xlabel='Cholesterol', ylabel='RestingBP'>

sns.set(rc = {'figure.figsize':(20,10)})
sns.heatmap(data.corr(), annot=True) #By saying annot=True it writes the correlation in each cell

<AxesSubplot:>

💡 Findings

We see that age and heart disease doesn't seem to have a strong relationship.
But there is a slightly notible negative correlation between MaxHeartRate and HeartDisease.
This means that as the MaxHeartRate increases the chance of heart disease decreases.
The exact opposite could also be noticed with the Oldpeak.

A problem we have here is that every feature here is numeric, so we are completely disregarding the categorical features. (Apart from heart disease)
So to quickly fix that we will turn the categorical features into dummy variables. </font>

cat_cols.remove('Sex')
cat_cols.remove('FastingBS')
dataFrameOne = pd.get_dummies(data, columns = cat_cols, drop_first = False)
dataFrameOne = pd.get_dummies(dataFrameOne, columns = ['Sex'], drop_first = True)
cat_cols.append('Sex')
cat_cols.append('FastingBS')
dataFrameOne

	Age	RestingBP	Cholesterol	FastingBS	MaxHR	Oldpeak	HeartDisease	ChestPainType_ASY	ChestPainType_ATA	ChestPainType_NAP	ChestPainType_TA	RestingECG_LVH	RestingECG_Normal	RestingECG_ST	ExerciseAngina_N	ExerciseAngina_Y	ST_Slope_Down	ST_Slope_Flat	ST_Slope_Up	Sex_M
0	40	140.0	289.0	0	172	0.000000	0	0	1	0	0	0	1	0	1	0	0	0	1	1
1	49	160.0	180.0	0	156	1.000000	1	0	0	1	0	0	1	0	1	0	0	1	0	0
2	37	130.0	283.0	0	98	0.000000	0	0	1	0	0	0	0	1	1	0	0	0	1	1
3	48	138.0	214.0	0	108	1.500000	1	1	0	0	0	0	1	0	0	1	0	1	0	0
4	54	150.0	195.0	0	122	0.000000	0	0	0	1	0	0	1	0	1	0	0	0	1	1
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
913	45	110.0	264.0	0	132	1.200195	1	0	0	0	1	0	1	0	1	0	0	1	0	1
914	68	144.0	193.0	1	141	3.400391	1	1	0	0	0	0	1	0	1	0	0	1	0	1
915	57	130.0	131.0	0	115	1.200195	1	1	0	0	0	0	1	0	0	1	0	1	0	1
916	57	130.0	236.0	0	174	0.000000	1	0	1	0	0	1	0	0	1	0	0	1	0	0
917	38	138.0	175.0	0	173	0.000000	0	0	0	1	0	0	1	0	1	0	0	0	1	1

746 rows × 20 columns

sns.pairplot(data)

<seaborn.axisgrid.PairGrid at 0x1b0f13cf160>

plt.figure(figsize=(15,10))
for i,col in enumerate(data.columns,1):
    plt.subplot(4,3,i) #4 rows, 3 columns, and i is the number of the subplot  
    plt.title(f"Distribution of {col} Data")
    sns.histplot(data[col],kde=True) #Setting Kernel Density Estimation (KDE) to True will draw a line, smoothing the distribution.
    plt.tight_layout()
    plt.plot()

sns.set(rc = {'figure.figsize':(25,12)})
sns.heatmap(dataFrameOne.corr(), annot=True) #We pass the correlation

<AxesSubplot:>

💡 Findings

Looking at the correlations we see a somewhat notible negative correlation between MaxHR and HeartDisease.
As well as notible positive correlation between Oldpeak and HeartDisease.
A interestingly high positive correlation between Asymptomatic chest pains and HeartDisease.
An interestinly high negative correlation between ATA chest pains and HeartDisease.

A very notable correlation between both Exercise Anginas and HeartDisease.
We notice that N and Y have the same correlation but one is positive and one is negative.

A similar thing happens with slope down and slope flat. </font>

fig=px.histogram(data, 
                 x="HeartDisease",
                 color="Sex",          
                 hover_data=data.columns,
                 title="Heart Disease for Males and Females",
                 barmode="group")

fig.write_image("Graphs/histogram.png", format="png", width=800, height=600, scale=2)
fig.show()

💡 Findings

We notice something very quickly here.
There seems to be a lot more males than females in this graph.
That throws off the accuracy of what we are trying to see.
It would be difficult to get a clear picture when there are just so many more males than females.


Lets take a look at how many more males there are.
To do this we make the x variable look at Sex instead of HeartDisease

</font>

fig=px.histogram(data, 
                 x="Sex",           
                 hover_data=data.columns,
                 title="Count for Males and Females",
                 barmode="group")
fig.show()

import plotly.express as px

fig = px.histogram(data, 
                   x="Sex",
                   color="HeartDisease",          
                   hover_data=data.columns,
                   title="Distribution of Heart Diseases for Male and Females",
                   barmode="group")

# Define the base font size that will be scaled according to the image size
base_font_size = 20

# Set the width and height directly within the figure using update_layout
fig.update_layout(width=980, height=560)

# Calculate the scale factor based on the desired image size
scale_factor = min(fig['layout']['width'] / 980, fig['layout']['height'] / 560)

# Calculate the adjusted font size
adjusted_title_font_size = int(base_font_size * scale_factor)
adjusted_axis_label_font_size = int(base_font_size * 0.8 * scale_factor)
adjusted_legend_font_size = int(base_font_size * 0.8 * scale_factor)

# Increase the text size for the title, axis labels, and legend
fig.update_layout(title_font=dict(size=adjusted_title_font_size),
                  xaxis_title_font=dict(size=adjusted_axis_label_font_size),
                  yaxis_title_font=dict(size=adjusted_axis_label_font_size),
                  legend_font=dict(size=adjusted_legend_font_size))

fig.write_image("Graphs/histogramMTry3.png", format="png", width=2560, height=1440, scale=1)
fig.show()

💡 Findings

Now this is very interesting, we see that the majority of Males tested had heart disease while the majority of females tested did not have heart disease.
It looks like men where significantly more likely to have heart disease.

If we assume that the data is collected when someone goes to the doctor we can come to interesting conclusions from this.
There are two things that may be true from this.

- Men may have a higher chance of getting Heart Disease.

and

- Men may be less inclined to go to the doctor until they are sure something is wrong, while women may be more willing to go to the doctor at the first sign of something wrong.
-- This means that women may go in when they don't need to, while men only go in when they need to.
--- Which results in more false alarms from women and less false alarms from men.
---- There is something else in the dataset we can look at to get an idea of if this is the case.

To see if men only go when they are absolutely sure something is wrong we can look at the ChestPainType category.
If there is not a large amount of asymptomatic records from the men and a lot of asymptomatic records from the women then it backs up the previous claim.

</font>

fig=px.histogram(data, 
                 x="Sex",
                 color="ChestPainType",           
                 hover_data=data.columns,
                 title="Distribution of Chest pains for males and females",
                 barmode="group")

fig.write_image("Graphs/histogramChest.png", format="png", width=800, height=600, scale=3)
fig.show()

💡 Findings

I reject my hypothesis.

We see the complete opposite from what I previously guessed.
We see that the large majority of male records where asymptomatic, while the majority of female records was also asymptomatic, but to a significantly smaller degree.

From this we can guess that the data is from a routine checkup and that men seem more inclined to go for said checkups.
We come to this conclusion because why would a man who has no symptoms go to the doctors.

But there is a problem with this conclusion again.

If this was just a sample of random people coming for a checkup, does that mean that the vast majority of men suffer from heart disease?

It doesn't make much sense, or it's an extremely scary find.

There is more evidence to support that this may be the case.
If we look at the distribution of ages, we see that the majority of ages are within the mid-life stages.
Around the time that people generally start going for routine checkups.

We will use matplotlib to plot the data.
Let's create a box and whisker diagram to see how the data is distributed.
And a histogram to see the curve and distribution of the data.

</font>

import matplotlib.pyplot as plt

'''

---------------------------------------------------------------------------------------------------------------------------

We want to split the data into two groups.
Men and woman and study them seperately to see if there is a difference.

------

'''
menData = data[data['Sex'] == 'M']
femaleData = data[data['Sex'] == 'F']

def createPlot(data,gender):
    sns.set(style="darkgrid") #Set the style of the graph using seaborn
    f, (ax_box, ax_hist) = plt.subplots(2, sharex=True, gridspec_kw={"height_ratios": (.15, .85)}) #We want to plot two graphs on the same figure
    sns.boxplot(data=data,x='Age', ax=ax_box) #First one is a box and whisker diagram
    sns.histplot(data=data, x="Age", ax=ax_hist,kde=True) #Second one is a histogram
    ax_box.set(xlabel='')
    ax_box.set(title=gender)
    plt.show()

createPlot(menData,'Men')
createPlot(femaleData,'Women')

print(f'''
      Men records: 
      {menData['Age'].describe()}
      ''')
print(f'''
      Females records: 
      {femaleData['Age'].describe()}
      ''')

      Men records: 
      count    564.000000
mean      53.102837
std        9.556603
min       28.000000
25%       46.000000
50%       54.000000
75%       59.000000
max       77.000000
Name: Age, dtype: float64
      

      Females records: 
      count    182.000000
mean      52.197802
std        9.339774
min       30.000000
25%       45.250000
50%       53.000000
75%       58.750000
max       76.000000
Name: Age, dtype: float64
      

💡 Findings

Most of the records are from people in their 40s to 60s with not much variation between men and women.
This supports the concept that these are from regular checkups as most people would be going around that age.

But there is one more variable we can look at.

Let's look at the ExerciseAngina variable.
Which is simply if they had pain when they exercised.

</font>

fig=px.histogram(data, 
                 x="ExerciseAngina",
                 color="ChestPainType",           
                 hover_data=data.columns,
                 title="Distribution of Chest pains ExerciseAngina",
                 barmode="group")
fig.write_image("Graphs/histogramExercise1.png", format="png", width=800, height=600, scale=3)
fig.show()

fig=px.histogram(data, 
                 x="ExerciseAngina",
                 color="HeartDisease",           
                 hover_data=data.columns,
                 title="Heart disease based on exercise anigna",
                 barmode="group")
fig.write_image("Graphs/histogramExercise.png", format="png", width=800, height=600, scale=3)
fig.show()

💡 Findings

It all makes sense now.
At first you think, how could people who experienced chest pains from exercise be asymptomatic.
It's because they ONLY experience chest pains from exercise.
Passively they don't feel much pain, so they say they don't have chest pain for the ChestPainType category.
But they do have chest pain when they exercise.

---

So what happens is that a man will go gym.
Do hardcore exercise.
Experience chest pains.
And then go to the doctor about it.

---

Doctor : "Why are you here"
Patient : "I had chest pains while exercising"
Doctor : Writes down Execise-induced-Angina , "Does your chest hurt at the moment?"
Patient : "No"
Doctor : Writes down asymptomatic.

---

We also see that the majority of Exercise angina cases result in Heart disease, while the majority that didn't experience and exercise
Angina do not have Heart disease.
We also see that there are still some cases of people with heart disease that didn't experience exercise pains but still had heart disease.
These could just be people that didn't exercise at all.

But the data suggests that the stress from exercise could be an early warning sign of heart problems.

---

It's a shot in the dark but it could also explain why there are more men than women.
As women may be less likely to put themselves under that stress that level of physical stress.

</font>

fig=px.histogram(data, 
                 x="ChestPainType",
                 color="HeartDisease",           
                 hover_data=data.columns,
                 title="Distribution of Heart Diseases",
                 barmode="group")
fig.write_image("Graphs/histogramHeart2.jpg", format="jpg", width=800, height=600, scale=3)

fig.show()

💡 Findings

An extremely interesting observation is that the people with chest pains typically don't seem to have heart disease.
But the people who don't have chest pains do seem to be the ones with heart disease.

I'm not a doctor and have little knowedge about this but I would definitely start asking questions along the lines of:
-Does heart disease maybe prevent circulation to your chest, resulting in potentially a lack of feeling there.
-Is chest pain caused by strong circulation, which isn't present with heart disease.

All I know is that this is something a researcher would be interested in.

Let's now compare ages to heart disease.
We can use a box and whisker for this.

</font>

fig = px.box(data,y="Age",x="HeartDisease",color="Sex",title=f"Distrubution of Age")
fig.write_image("Graphs/ageGraph.jpg", format="jpg", width=800, height=600, scale=3)
fig.show()

💡 Findings

Unsuprisingly, the average age seems to be higher for people who have heart disease.
With no real difference between women and men.
This makes sense as your heart gets weaker and more damaged over time.

Let's take a look at the ECG variable next.

</font>

fig=px.histogram(data,
                 x="RestingECG",
                 color="HeartDisease",
                 hover_data=data.columns,
                 title="Heart Diseases by ECG",
                 barmode="group")
fig.show()

💡 Findings

We see that the ECG doesn't seem to matter too much.
You can make a case that the there is always a higher chance of heart disease regradless.
But that's because there are more records with heart disease compared to records without heart disease.

It is however noticible that the ST category does seem to matter a little bit with a high concentration of positive heart diseases.
But it's not by much.
Especially since the majority of the data leans towards a normal ECG regardless.

Next we can look at cholesterol levels.
To visualise this we can look at scatterplots for people who have heart disease and people that don't

</font>

fig = px.box(data,y="Cholesterol",x="HeartDisease",color="HeartDisease",title=f"Distrubution of Age")
fig.write_image("Graphs/chol2.jpg", format="jpg", width=600, height=440, scale=1)
fig.show()

fig=px.histogram(data, 
                 x="HeartDisease",
                 color="FastingBS",           
                 hover_data=data.columns,
                 title="Fasting Blood Sugar vs Heart Disease", 
                 barmode="group")
fig.show()

fig=px.histogram(data, 
                 x="FastingBS",
                 color="HeartDisease",           
                 hover_data=data.columns,
                 title="Heart Disease vs Fasting Blood Sugar", 
                 barmode="group")
fig.show()

fig = px.box(data,y="MaxHR",x="HeartDisease",color="HeartDisease",title=f"Max Heart Rate and Heart Disease Scatterplot")
fig.write_image("Graphs/ageThingYes.jpg", format="jpg", width=800, height=600, scale=3)
fig.show()

fig = px.box(data,y="RestingBP",x="HeartDisease",color="HeartDisease",title=f"Resting Blood Pressure and Heart Disease Scatterplot")
fig.write_image("Graphs/bloodPresure.jpg", format="jpg", width=800, height=600, scale=3)
fig.show()

fig = px.box(data,y="Oldpeak",x="HeartDisease",color="HeartDisease",title=f"Oldpeak and Heart Disease Scatterplot")
fig.write_image("Graphs/bloodPresure123.jpg", format="jpg", width=800, height=600, scale=3)
fig.show()

💡 Findings

Now it seems quite interesting.
When the oldpeak is 0 your chance of heart disease is suprisingly low.
But if your oldpeak is > 0 your chance of heart disease is significantly higher.

The last variable to check is the ST slope.

</font>

fig=px.histogram(data, 
                 x="ST_Slope",
                 color="HeartDisease",
                 hover_data=data.columns,
                 title="Slope vs Heart Disease",
                 barmode="group")
fig.write_image("Graphs/bloodPresureslope.jpg", format="jpg", width=800, height=600, scale=3)
fig.show()

💡 BIG FIND

BIG FIND: If your ST_Slope is going up then you have a dramatically reduced chance of having heart disease.
But if it's flat or down then the opposite is true.

Let's see how the numerical data is grouped together with regards to heart disease.
Paying special attention to the distribution graphs going down the middle diagonally.

</font>

sns.pairplot(data, hue="HeartDisease")

<seaborn.axisgrid.PairGrid at 0x1b1017e63b0>

💡 Findings

We see some definite distinctions here with Oldpeak, MaxHR, Age and FastingBS
Where they so grouping based on the outcome of the heart disease.

</font>

Image(filename='Images/grouping2.png') 

💡 CONCLUSION

We gained a lot from this data analysis.
-We found out that men are more likely to get heart disease.
-We found out that exercise is a good way to know if you could have heart disease.
-We found out that heart disease doesn't cause Chest pain passively.
-We found out that heart disease is more apparent in older people.
-We found out that an Oldpeak of > 0 increases your chance of having heart disease.
-We found out that a higher MaxHR lowers your chance of having heart disease.

</font>

👨‍💻 MODEL CREATION

💡 Info

The next thing we should do is to break the data into an X variable and a y variable.
The X variable is the independent varaible and the y variable is the dependent variable.
In this case the y varaible is out target. Which is HeartDisease.
For the sake of accuracy our y variable will be everything that isn't the target.

</font>

X = dataFrameOne.drop(['HeartDisease'],axis=1)
y = dataFrameOne['HeartDisease']

print(f'''
      The shape of X is {X.shape} and the type is {type(X)}
      The shape of y is {y.shape} and the type is {type(y)}
      ''')

      The shape of X is (746, 19) and the type is <class 'pandas.core.frame.DataFrame'>
      The shape of y is (746,) and the type is <class 'pandas.core.series.Series'>
      

💡 Findings

We see there are 746 records for both the X and y variables.
We see that the X variable has 19 rows while the y variable is flat.
The y variable is flat because it is a pandas series.

</font>

X.head()

	Age	RestingBP	Cholesterol	FastingBS	MaxHR	Oldpeak	ChestPainType_ASY	ChestPainType_ATA	ChestPainType_NAP	ChestPainType_TA	RestingECG_LVH	RestingECG_Normal	RestingECG_ST	ExerciseAngina_N	ExerciseAngina_Y	ST_Slope_Down	ST_Slope_Flat	ST_Slope_Up	Sex_M
0	40	140.0	289.0	0	172	0.0	0	1	0	0	0	1	0	1	0	0	0	1	1
1	49	160.0	180.0	0	156	1.0	0	0	1	0	0	1	0	1	0	0	1	0	0
2	37	130.0	283.0	0	98	0.0	0	1	0	0	0	0	1	1	0	0	0	1	1
3	48	138.0	214.0	0	108	1.5	1	0	0	0	0	1	0	0	1	0	1	0	0
4	54	150.0	195.0	0	122	0.0	0	0	1	0	0	1	0	1	0	0	0	1	1

y.head()

0    0
1    1
2    0
3    1
4    0
Name: HeartDisease, dtype: int8

💡 Findings

Looks good.

Let's scale down the numerical data.
This will help us to build a model that is more accurate and faster.
It will take all the numbers and scale them down to a range of 0 to 1.

For this we will use sklearn's MinMaxScaler and we will only scale the relevant columns.

</font>

scaler = MinMaxScaler()
X[num_cols] = scaler.fit_transform(X[num_cols])
print(f'''
      The columns affected will be {num_cols}''')
X.head()

      The columns affected will be ['Age', 'RestingBP', 'Cholesterol', 'MaxHR', 'Oldpeak']

	Age	RestingBP	Cholesterol	FastingBS	MaxHR	Oldpeak	ChestPainType_ASY	ChestPainType_ATA	ChestPainType_NAP	ChestPainType_TA	RestingECG_LVH	RestingECG_Normal	RestingECG_ST	ExerciseAngina_N	ExerciseAngina_Y	ST_Slope_Down	ST_Slope_Flat	ST_Slope_Up	Sex_M
0	0.244898	0.444444	0.393822	0	0.774436	0.015871	0	1	0	0	0	1	0	1	0	0	0	1	1
1	0.428571	0.629630	0.183398	0	0.654135	0.174622	0	0	1	0	0	1	0	1	0	0	1	0	0
2	0.183673	0.351852	0.382239	0	0.218045	0.015871	0	1	0	0	0	0	1	1	0	0	0	1	1
3	0.408163	0.425926	0.249035	0	0.293233	0.253997	1	0	0	0	0	1	0	0	1	0	1	0	0
4	0.530612	0.537037	0.212355	0	0.398496	0.015871	0	0	1	0	0	1	0	1	0	0	0	1	1

💡 Findings

Looks like it all went well.

Let's visualise what happened. Let's look at age before and after being scaled.

</font>

'''
BEFORE BEING SCALED:
'''

plt.figure(figsize=(10,5))
sns.histplot(data['Age'],kde=True) 

<AxesSubplot:xlabel='Age', ylabel='Count'>

'''
AFTER IT WAS SCALED:
'''
plt.figure(figsize=(10,5))
sns.histplot(X['Age'],kde=True)

<AxesSubplot:xlabel='Age', ylabel='Count'>

💡 Findings

We see that the distribution of the data has not changed at all. <-- Important to note. Instead what has happened it that the scale of the X axis has been changed from being between 30 and 70 To being between 0 and 1.

The next step is to split the data into a training set and a test set. We will NOT train the data using any of the testing data.

We will use the sklearn library to import the train_test_split function. Which will automatically breakup our data. For now we will use 80% training set and 20% test set. And a random state of 42 to keep it consistent.

We are also not going to work with any scaled data. The reason is that we are going to use a pipeline to scale the data anyway. So this let's us analyse unscaled data but build the model using scaled data automatically.

</font>

X = dataFrameOne.drop(['HeartDisease'],axis=1)
y = dataFrameOne['HeartDisease']

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state=42) 

X_train.describe()

	Age	RestingBP	Cholesterol	FastingBS	MaxHR	Oldpeak	ChestPainType_ASY	ChestPainType_ATA	ChestPainType_NAP	ChestPainType_TA	RestingECG_LVH	RestingECG_Normal	RestingECG_ST	ExerciseAngina_N	ExerciseAngina_Y	ST_Slope_Down	ST_Slope_Flat	ST_Slope_Up	Sex_M
count	596.000000	596.000000	596.000000	596.000000	596.000000	596.000000	596.000000	596.000000	596.000000	596.000000	596.000000	596.000000	596.000000	596.000000	596.000000	596.000000	596.000000	596.000000	596.000000
mean	52.837248	132.741611	245.050336	0.157718	140.256711	0.905273	0.498322	0.226510	0.224832	0.050336	0.231544	0.607383	0.161074	0.622483	0.377517	0.052013	0.476510	0.471477	0.750000
std	9.425445	17.095688	58.728495	0.364783	24.552182	1.062500	0.500417	0.418924	0.417822	0.218820	0.422173	0.488743	0.367908	0.485173	0.485173	0.222241	0.499867	0.499605	0.433376
min	29.000000	92.000000	85.000000	0.000000	69.000000	-0.099976	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
25%	46.000000	120.000000	208.750000	0.000000	122.750000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.750000
50%	54.000000	130.000000	236.500000	0.000000	140.000000	0.500000	0.000000	0.000000	0.000000	0.000000	0.000000	1.000000	0.000000	1.000000	0.000000	0.000000	0.000000	0.000000	1.000000
75%	59.000000	140.000000	275.000000	0.000000	160.000000	1.500000	1.000000	0.000000	0.000000	0.000000	0.000000	1.000000	0.000000	1.000000	1.000000	0.000000	1.000000	1.000000	1.000000
max	77.000000	200.000000	603.000000	1.000000	202.000000	5.601562	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000

Random Forest Classification

💡 Findings

We see that 80% of the data has gone into the X_train and y_train variables. The remaining 20% will be in the X_test and y_test variables.

Let's first take a look at the random forest classification model.

First we import the nessicary classes. For this we will need the RandomForestClassifier method and the make_pipeline method. The pipeline will scale our data before training and testing the model.

</font>

ignoreWarnings()
rf_pipe = make_pipeline(scaler, RandomForestClassifier(n_estimators=4)) #We use 4 trees

rf_pipe.fit(X_train, y_train) #We fit the model

outputs = cross_val_score(rf_pipe, X_train, y_train, cv=5, scoring='accuracy') #We get outputs
averageScore_rf = outputs.mean() #We get the average score
y_pred = rf_pipe.predict(X_test) #We get the predictions
print(outputs)
print(f'The average score was {averageScore_rf}')

print(classification_report(y_test, y_pred))
plot_confusion_matrix(rf_pipe, X_test, y_test)
plt.grid(False)
plt.title('Random Forest Classifier before hyper parameters')

[0.83333333 0.8487395  0.79831933 0.76470588 0.81512605]
The average score was 0.8120448179271709
              precision    recall  f1-score   support

           0       0.79      0.93      0.85        71
           1       0.92      0.77      0.84        79

    accuracy                           0.85       150
   macro avg       0.85      0.85      0.85       150
weighted avg       0.86      0.85      0.85       150

Text(0.5, 1.0, 'Random Forest Classifier before hyper parameters')

#Number of trees 
n_estimators = [int(x) for x in np.linspace(start = 10, stop = 80, num = 10)]

#Number of features to consider at every split
max_features = ['auto', 'sqrt']

#Maximum number of levels in tree
max_depth = [2,4]

#Minimum number of samples required to split a node
min_sample_split = [2, 5]

#Minimum number of samples required at each leaf node
min_sample_leaf = [1, 2]

#Method of selecting samples for training each tree
bootstrap = [True, False]

param_grid = [{'n_estimators': n_estimators, 
               'max_features': max_features, 
               'max_depth': max_depth, 
               'min_samples_split': min_sample_split, 
               'min_samples_leaf': min_sample_leaf, 
               'bootstrap': bootstrap}]

print(param_grid)

[{'n_estimators': [10, 17, 25, 33, 41, 48, 56, 64, 72, 80], 'max_features': ['auto', 'sqrt'], 'max_depth': [2, 4], 'min_samples_split': [2, 5], 'min_samples_leaf': [1, 2], 'bootstrap': [True, False]}]

rf_Grid = GridSearchCV(estimator = RandomForestClassifier(), param_grid = param_grid, cv = 5, n_jobs = 4, verbose = 2)#
rf_Grid.fit(X_train, y_train)
bestParams = rf_Grid.best_params_
print(f'''
      n_estimators: {bestParams["n_estimators"]}
      min_samples_split: {bestParams["min_samples_split"]}
      min_samples_leaf: {bestParams["min_samples_leaf"]}
      max_features: {bestParams["max_features"]}
      max_depth: {bestParams["max_depth"]}
      bootstrap: {bestParams["bootstrap"]}''')

Fitting 5 folds for each of 320 candidates, totalling 1600 fits

      n_estimators: 56
      min_samples_split: 5
      min_samples_leaf: 2
      max_features: sqrt
      max_depth: 4
      bootstrap: False

rf_pipe = make_pipeline(scaler, RandomForestClassifier(n_estimators=bestParams['n_estimators'], max_features=bestParams['max_features'], max_depth=bestParams['max_depth'], min_samples_split=bestParams['min_samples_split'], min_samples_leaf=bestParams['min_samples_leaf'], bootstrap=bestParams['bootstrap']))

rf_pipe.fit(X_train, y_train)
averageScore_rf = cross_val_score(rf_pipe, X_train, y_train, cv=5, scoring='accuracy')
print(averageScore_rf)
print(averageScore_rf.mean())

y_pred = rf_pipe.predict(X_test)
finalAccuracy_rf = accuracy_score(y_test, y_pred)
print(finalAccuracy_rf)

[0.81666667 0.85714286 0.87394958 0.80672269 0.86554622]
0.8440056022408964
0.9

print(classification_report(y_test, y_pred))
plot_confusion_matrix(rf_pipe, X_test, y_test)
plt.grid(False)
plt.title('Random Forest Classifier')

              precision    recall  f1-score   support

           0       0.89      0.90      0.90        71
           1       0.91      0.90      0.90        79

    accuracy                           0.90       150
   macro avg       0.90      0.90      0.90       150
weighted avg       0.90      0.90      0.90       150

Text(0.5, 1.0, 'Random Forest Classifier')

💡 Findings

The model scored very well across the board and the confusion matrix shows 63 correct falses and 70 correct trues. (During my testing) There were however 6 cases where the model said the person did NOT have heart disease when they actually did. This is significantly better than the 15 before hyper parameters. However, this incorrect prediction could have costed them their lives. So this model is to be taken with a grain of salt.

</font>

def featureImportance(model,label):
    feature_col_tree = X.columns.to_list()
    
    X_train_scaled = scaler.fit_transform(X_train)
    model.fit(X_train_scaled, y_train)
    plt.figure(figsize=(20,15))
    importance = model.feature_importances_
    idxs = np.argsort(importance)
    plt.title("Feature Importance")
    plt.barh(range(len(idxs)),importance[idxs],align="center")
    plt.yticks(range(len(idxs)),[feature_col_tree[i] for i in idxs])
    plt.xlabel(label)
    plt.show()

model = RandomForestClassifier(n_estimators=bestParams['n_estimators'], max_features=bestParams['max_features'], max_depth=bestParams['max_depth'], min_samples_split=bestParams['min_samples_split'], min_samples_leaf=bestParams['min_samples_leaf'], bootstrap=bestParams['bootstrap'])
featureImportance(model, "Feature Importance for Random Forest Classifier")

💡 Findings

Looks like the slope variables are the most important. (Except ST_Slope_Down) It's interesting to look at these variables and see if our data analysis was correct. So far it looks like it was.

</font>

Naive Bayes Classifier

nb_pipe = make_pipeline(scaler, GaussianNB(var_smoothing=50)) 

nb_pipe.fit(X_train, y_train) #We fit the model

outputs = cross_val_score(nb_pipe, X_train, y_train, cv=5, scoring='accuracy') #We get outputs
averageScore = outputs.mean() #We get the average score

print(outputs)
print(f'The average score was {averageScore}')

y_pred = nb_pipe.predict(X_test) #We get the predictions
print(classification_report(y_test, y_pred))
plot_confusion_matrix(nb_pipe, X_test, y_test)
plt.grid(False)

[0.61666667 0.54621849 0.58823529 0.6302521  0.6302521 ]
The average score was 0.6023249299719888
              precision    recall  f1-score   support

           0       0.52      1.00      0.69        71
           1       1.00      0.18      0.30        79

    accuracy                           0.57       150
   macro avg       0.76      0.59      0.49       150
weighted avg       0.77      0.57      0.48       150

#Widens or smooths the curve of the Gaussian distribution
var_smoothing = np.logspace(0,-9, num=100)

param_grid = [{'var_smoothing': var_smoothing}]

Nb_grid = GridSearchCV(estimator = GaussianNB(), param_grid = param_grid, cv = 5, n_jobs = 4, verbose = 2)#
Nb_grid.fit(X_train, y_train)
bestParams = Nb_grid.best_params_
print(f'''
      Smoothing: {bestParams["var_smoothing"]}
      ''')

Fitting 5 folds for each of 100 candidates, totalling 500 fits

      Smoothing: 2.310129700083158e-05
      

nb_pipe = make_pipeline(scaler, GaussianNB(var_smoothing=bestParams['var_smoothing']))
nb_pipe.fit(X_train, y_train)
averageScore_nb = cross_val_score(nb_pipe, X_train, y_train, cv=5, scoring='accuracy')
print(averageScore_nb)
print(averageScore_nb.mean())

y_pred = nb_pipe.predict(X_test)
finalAccuracy_nb = accuracy_score(y_test, y_pred)
print(finalAccuracy_nb)

[0.84166667 0.8487395  0.86554622 0.80672269 0.84033613]
0.8406022408963585
0.8666666666666667

print(classification_report(y_test, y_pred))
plot_confusion_matrix(nb_pipe, X_test, y_test)
plt.grid(False)
plt.title('Naive bayes')

              precision    recall  f1-score   support

           0       0.85      0.87      0.86        71
           1       0.88      0.86      0.87        79

    accuracy                           0.87       150
   macro avg       0.87      0.87      0.87       150
weighted avg       0.87      0.87      0.87       150

Text(0.5, 1.0, 'Naive bayes')

💡 Findings

The model is pretty good. But not as good as the Random Forest model.

</font>

K-Nearest Neighbour

knn_pipe = make_pipeline(scaler, KNeighborsClassifier(n_neighbors=2))
knn_pipe.fit(X_train, y_train) #We fit the model
outputs = cross_val_score(knn_pipe, X_train, y_train, cv=5, scoring='accuracy') #We get outputs
averageScore = outputs.mean() #We get the average score
print(outputs)
print(f'The average score was {averageScore}')

[0.81666667 0.77310924 0.79831933 0.78151261 0.84033613]
The average score was 0.8019887955182072

#Leaf size passed to BallTree or KDTree.
leaf_size = np.logspace(0,-9, num=100)

#Number of neighbors to use.
n_neighbors = np.arange(1,22,step=2)

'''
uniform : uniform weights. All points in each neighborhood are weighted equally.
distance : weight points by the inverse of their distance. in this case, closer neighbors of a query point will have a greater influence than neighbors which are further away.

weights = ['uniform', 'distance'] <-- We are going to leave these out because they take an hour to process

---------------------------------------------------------------------------------------------------------------------------

Algorithm used to compute the nearest neighbors

algorithm = ['auto', 'ball_tree', 'kd_tree', 'brute'] <-- We are going to leave these out because they take an hour to process
'''

p = [1,2]

param_grid = [{'p': p, 'leaf_size': leaf_size, 'n_neighbors': n_neighbors}]
print(param_grid)

[{'p': [1, 2], 'leaf_size': array([1.00000000e+00, 8.11130831e-01, 6.57933225e-01, 5.33669923e-01,
       4.32876128e-01, 3.51119173e-01, 2.84803587e-01, 2.31012970e-01,
       1.87381742e-01, 1.51991108e-01, 1.23284674e-01, 1.00000000e-01,
       8.11130831e-02, 6.57933225e-02, 5.33669923e-02, 4.32876128e-02,
       3.51119173e-02, 2.84803587e-02, 2.31012970e-02, 1.87381742e-02,
       1.51991108e-02, 1.23284674e-02, 1.00000000e-02, 8.11130831e-03,
       6.57933225e-03, 5.33669923e-03, 4.32876128e-03, 3.51119173e-03,
       2.84803587e-03, 2.31012970e-03, 1.87381742e-03, 1.51991108e-03,
       1.23284674e-03, 1.00000000e-03, 8.11130831e-04, 6.57933225e-04,
       5.33669923e-04, 4.32876128e-04, 3.51119173e-04, 2.84803587e-04,
       2.31012970e-04, 1.87381742e-04, 1.51991108e-04, 1.23284674e-04,
       1.00000000e-04, 8.11130831e-05, 6.57933225e-05, 5.33669923e-05,
       4.32876128e-05, 3.51119173e-05, 2.84803587e-05, 2.31012970e-05,
       1.87381742e-05, 1.51991108e-05, 1.23284674e-05, 1.00000000e-05,
       8.11130831e-06, 6.57933225e-06, 5.33669923e-06, 4.32876128e-06,
       3.51119173e-06, 2.84803587e-06, 2.31012970e-06, 1.87381742e-06,
       1.51991108e-06, 1.23284674e-06, 1.00000000e-06, 8.11130831e-07,
       6.57933225e-07, 5.33669923e-07, 4.32876128e-07, 3.51119173e-07,
       2.84803587e-07, 2.31012970e-07, 1.87381742e-07, 1.51991108e-07,
       1.23284674e-07, 1.00000000e-07, 8.11130831e-08, 6.57933225e-08,
       5.33669923e-08, 4.32876128e-08, 3.51119173e-08, 2.84803587e-08,
       2.31012970e-08, 1.87381742e-08, 1.51991108e-08, 1.23284674e-08,
       1.00000000e-08, 8.11130831e-09, 6.57933225e-09, 5.33669923e-09,
       4.32876128e-09, 3.51119173e-09, 2.84803587e-09, 2.31012970e-09,
       1.87381742e-09, 1.51991108e-09, 1.23284674e-09, 1.00000000e-09]), 'n_neighbors': array([ 1,  3,  5,  7,  9, 11, 13, 15, 17, 19, 21])}]

knn_grid = GridSearchCV(estimator = KNeighborsClassifier(), param_grid = param_grid, cv = 5, n_jobs = 4, verbose = 2)#
knn_grid.fit(X_train, y_train)
bestParams = knn_grid.best_params_
print(f'''
      p: {bestParams["p"]}
      leaf_size: {bestParams["leaf_size"]}
      n_neighbors: {bestParams["n_neighbors"]}     
      ''')

Fitting 5 folds for each of 2200 candidates, totalling 11000 fits

      p: 1
      leaf_size: 1.0
      n_neighbors: 17     
      

knn_pipe = make_pipeline(scaler, KNeighborsClassifier(p=bestParams['p'], leaf_size=bestParams['leaf_size'], n_neighbors=bestParams['n_neighbors']))
knn_pipe.fit(X_train, y_train)
averageScore_knn = cross_val_score(knn_pipe, X_train, y_train, cv=5, scoring='accuracy').mean()
print(averageScore_knn)

0.8489775910364145

y_pred = knn_pipe.predict(X_test)
finalAccuracy_knn = accuracy_score(y_test, y_pred)

print(classification_report(y_test, y_pred))
plot_confusion_matrix(knn_pipe, X_test, y_test)
plt.grid(False)
plt.title('K-nearest Neighbour')

              precision    recall  f1-score   support

           0       0.88      0.89      0.88        71
           1       0.90      0.89      0.89        79

    accuracy                           0.89       150
   macro avg       0.89      0.89      0.89       150
weighted avg       0.89      0.89      0.89       150

Text(0.5, 1.0, 'K-nearest Neighbour')

Support Vector Machines

svc_pipe = make_pipeline(scaler, svm.SVC(C=20))
svc_pipe.fit(X_train, y_train) #We fit the model
outputs = cross_val_score(svc_pipe, X_train, y_train, cv=5, scoring='accuracy') #We get outputs
averageScore = outputs.mean() #We get the average score
print(outputs)
print(f'The average score was {averageScore}')

[0.825      0.84033613 0.8487395  0.74789916 0.87394958]
The average score was 0.8271848739495798

#Regularization parameter. The strength of the regularization is inversely proportional to C.
C = [0.1, 1, 10, 100, 1000]

#Kernel coefficient for ‘rbf’, ‘poly’ and ‘sigmoid’
gamma = [1, 0.1, 0.01, 0.001, 0.0001]

#Specifies the kernel type to be used in the algorithm.
kernel = ['rbf']

param_grid = [{'C': C, 'gamma': gamma, 'kernel': kernel}]
print(param_grid)

[{'C': [0.1, 1, 10, 100, 1000], 'gamma': [1, 0.1, 0.01, 0.001, 0.0001], 'kernel': ['rbf']}]

svc_grid = GridSearchCV(estimator = svm.SVC(), param_grid = param_grid, cv = 5, n_jobs = 4, verbose = 2)#
svc_grid.fit(X_train, y_train)
bestParams = svc_grid.best_params_
print(f'''
      C: {bestParams["C"]}
      gamma: {bestParams["gamma"]}
      kernel: {bestParams["kernel"]}
      ''')

Fitting 5 folds for each of 25 candidates, totalling 125 fits

      C: 100
      gamma: 0.0001
      kernel: rbf
      

svc_pipe = make_pipeline(scaler, svm.SVC(C=bestParams['C'], gamma=bestParams['gamma'], kernel=bestParams['kernel']))
svc_pipe.fit(X_train, y_train)
averageScore_svc = cross_val_score(svc_pipe, X_train, y_train, cv=5, scoring='accuracy').mean()
print(averageScore_svc)

0.8456442577030812

y_pred = svc_pipe.predict(X_test)
finalAccuracy_svc = accuracy_score(y_test, y_pred)
print(classification_report(y_test, y_pred))
plot_confusion_matrix(svc_pipe, X_test, y_test)
plt.grid(False)
plt.title('SVC')

              precision    recall  f1-score   support

           0       0.90      0.89      0.89        71
           1       0.90      0.91      0.91        79

    accuracy                           0.90       150
   macro avg       0.90      0.90      0.90       150
weighted avg       0.90      0.90      0.90       150

Text(0.5, 1.0, 'SVC')

dt_pipe = make_pipeline(scaler, DecisionTreeClassifier())
dt_pipe.fit(X_train, y_train) #We fit the model
outputs = cross_val_score(dt_pipe, X_train, y_train, cv=5, scoring='accuracy') #We get outputs
averageScore = outputs.mean() #We get the average score
print(outputs)
print(f'The average score was {averageScore}')

[0.78333333 0.77310924 0.83193277 0.68907563 0.77310924]
The average score was 0.770112044817927

#The maximum number of nodes the tree can have. The by limiting the nodes you can prevent overfitting.
max_depth = np.arange(1,200,step=10)

param_grid = [{'max_depth': max_depth}]
print(param_grid)

[{'max_depth': array([  1,  11,  21,  31,  41,  51,  61,  71,  81,  91, 101, 111, 121,
       131, 141, 151, 161, 171, 181, 191])}]

dt_grid = GridSearchCV(estimator = DecisionTreeClassifier(), param_grid = param_grid, cv = 5, n_jobs = 4, verbose = 2)#
dt_grid.fit(X_train, y_train)
bestParams = dt_grid.best_params_
print(f'''
        max_depth: {bestParams["max_depth"]}
      ''')

Fitting 5 folds for each of 20 candidates, totalling 100 fits

        max_depth: 1
      

dt_pipe = make_pipeline(scaler, DecisionTreeClassifier(max_depth=bestParams['max_depth']))
dt_pipe.fit(X_train, y_train)
averageScore_dt = cross_val_score(dt_pipe, X_train, y_train, cv=5, scoring='accuracy').mean()
print(averageScore_dt)

0.8154621848739495

y_pred = dt_pipe.predict(X_test)
finalAccuracy_dt = accuracy_score(y_test, y_pred)
print(classification_report(y_test, y_pred))
plot_confusion_matrix(dt_pipe, X_test, y_test)
plt.grid(False)
plt.title('DT')

              precision    recall  f1-score   support

           0       0.87      0.83      0.85        71
           1       0.85      0.89      0.87        79

    accuracy                           0.86       150
   macro avg       0.86      0.86      0.86       150
weighted avg       0.86      0.86      0.86       150

Text(0.5, 1.0, 'DT')

print(f'''
      Random Forest Average Accuracy {round(averageScore_rf.mean(),3)}
      Naive Bayes Average Accuracy {round(averageScore.mean(),3)}
      K-nearest Neighbour Average Accuracy {round(averageScore_knn.mean(),3)}
      SVC Average Accuracy {round(averageScore_svc.mean(),3)}
      Decision Tree Average Accuracy {round(averageScore_dt.mean(),3)}
      
      ---------------------------------------------------------------------------------------------------------------------------
      
      Random Forest Final Accuracy {round(finalAccuracy_rf.mean(),3)}
      Naive Bayes Final Accuracy {round(finalAccuracy_nb.mean(),3)}
      K-nearest Neighbour Final Accuracy {round(finalAccuracy_knn.mean(),3)}
      SVC Final Accuracy {round(finalAccuracy_svc.mean(),3)}
      Decision Tree Final Accuracy {round(finalAccuracy_dt.mean(),3)}
      
      ''')

      Random Forest Average Accuracy 0.844
      Naive Bayes Average Accuracy 0.77
      K-nearest Neighbour Average Accuracy 0.849
      SVC Average Accuracy 0.846
      Decision Tree Average Accuracy 0.815
      
      ---------------------------------------------------------------------------------------------------------------------------
      
      Random Forest Final Accuracy 0.9
      Naive Bayes Final Accuracy 0.867
      K-nearest Neighbour Final Accuracy 0.887
      SVC Final Accuracy 0.9
      Decision Tree Final Accuracy 0.86
      
      

💡 Findings

From my testing the SVC model and Random Forest model both performed the best

</font>

👨‍💻 ADVANCED MODEL CREATION 👨‍💻

class DropNull(BaseEstimator, TransformerMixin):
    def __init__(self):
        pass
    def fit(self, X, y=None):
        return self
    def transform(self, X, y=None):
        return X.dropna()

def oneHotEncodeSexDef(x):
    if x == 'M':
        return 1
    else:
        return 0

class oneHotEncodeSex(BaseEstimator, TransformerMixin):
    def __init__(self):
        pass
    def fit(self, X, y=None):
        return self
    def transform(self, X, y=None):
        X['Sex'] = X['Sex'].apply(oneHotEncodeSexDef)
        #Convert to int8
        X['Sex'] = X['Sex'].astype('uint8')
        
        return X

class getDummyValues(BaseEstimator, TransformerMixin):
    def __init__(self):
        pass
    def fit(self, X, y=None):
        return self
    def transform(self, X, y=None):
        catColumns = X.select_dtypes(include=['object']).columns
        X_withDummies = pd.get_dummies(X, columns=catColumns)
        return X_withDummies

class convertTargetToUint8(BaseEstimator, TransformerMixin):
    def __init__(self,target):
        self.target = target
    def fit(self, X, y=None):
        return self
    def transform(self, X, y=None):
        X[self.target] = X[self.target].astype('uint8')
        return X

df = pd.read_csv('Csv Files/heart.csv') 
df = df.dropna()

pipeline = make_pipeline(DropNull(),oneHotEncodeSex(),getDummyValues(),convertTargetToUint8('HeartDisease'))
newDf = pipeline.fit_transform(df)

#Get all columns that aren't int8
nonInt8Columns = newDf.select_dtypes(exclude=['uint8']).columns
scaler = MinMaxScaler()
newDf[nonInt8Columns] = scaler.fit_transform(newDf[nonInt8Columns])

ignoreWarnings()
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score, r2_score, roc_auc_score, precision_score, recall_score, f1_score
import pandas as pd
#Import SVC 


models = [RandomForestClassifier(), LogisticRegression(), KNeighborsClassifier(),
          DecisionTreeClassifier(), GradientBoostingClassifier(),
          CatBoostClassifier(logging_level='Silent'), XGBClassifier(logging_level='Silent'),
          LGBMClassifier(logging_level='Silent'), AdaBoostClassifier(), SVC(probability=True)]  # Make sure SVC has probability=True



def getScores(model, X, y):
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)
    model.fit(X_train, y_train)
    
    # If the model supports 'predict_proba', use it; otherwise, use 'decision_function'
    if hasattr(model, 'predict_proba'):
        y_pred_probs = model.predict_proba(X_test)[:, 1]
    else:
        y_pred_probs = model.decision_function(X_test)
    
    y_pred = model.predict(X_test)
    accuracy = accuracy_score(y_test, y_pred)
    r2 = r2_score(y_test, y_pred)
    roc_auc = roc_auc_score(y_test, y_pred_probs)
    
    precision_0 = precision_score(y_test, y_pred, pos_label=0)
    precision_1 = precision_score(y_test, y_pred, pos_label=1)
    
    recall_0 = recall_score(y_test, y_pred, pos_label=0)
    recall_1 = recall_score(y_test, y_pred, pos_label=1)
    
    f1 = f1_score(y_test, y_pred)
    
    return accuracy, r2, roc_auc, precision_0, precision_1, recall_0, recall_1, f1, model

diction = {
    'Model': [],
    'Accuracy': [],
    'R2': [],
    'ROC_AUC': [],
    'Precision_0': [],
    'Precision_1': [],
    'Recall_0': [],
    'Recall_1': [],
    'F1_Score': [],
    'Combined': []
}

for _ in range(5):
    for model in models:
        accuracy, r2, roc_auc, precision_0, precision_1, recall_0, recall_1, f1, model = getScores(model, newDf.drop('HeartDisease', axis=1), newDf['HeartDisease'])
        diction['Model'].append(model.__class__.__name__)
        diction['Accuracy'].append(accuracy)
        diction['R2'].append(r2)
        diction['ROC_AUC'].append(roc_auc)
        diction['Precision_0'].append(precision_0)
        diction['Precision_1'].append(precision_1)
        diction['Recall_0'].append(recall_0)
        diction['Recall_1'].append(recall_1)
        diction['F1_Score'].append(f1)
        diction['Combined'].append(accuracy + r2 + roc_auc)

reportDataframe = pd.DataFrame(diction)
reportDataframe_avg = reportDataframe.groupby('Model').mean().reset_index()

[17:02:14] WARNING: C:/Users/Administrator/workspace/xgboost-win64_release_1.5.1/src/learner.cc:576: 
Parameters: { "logging_level" } might not be used.

  This could be a false alarm, with some parameters getting used by language bindings but
  then being mistakenly passed down to XGBoost core, or some parameter actually being used
  but getting flagged wrongly here. Please open an issue if you find any such cases.


[17:02:14] WARNING: C:/Users/Administrator/workspace/xgboost-win64_release_1.5.1/src/learner.cc:1115: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[LightGBM] [Warning] Unknown parameter: logging_level
[17:02:17] WARNING: C:/Users/Administrator/workspace/xgboost-win64_release_1.5.1/src/learner.cc:576: 
Parameters: { "logging_level" } might not be used.

  This could be a false alarm, with some parameters getting used by language bindings but
  then being mistakenly passed down to XGBoost core, or some parameter actually being used
  but getting flagged wrongly here. Please open an issue if you find any such cases.


[17:02:17] WARNING: C:/Users/Administrator/workspace/xgboost-win64_release_1.5.1/src/learner.cc:1115: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[LightGBM] [Warning] Unknown parameter: logging_level
[17:02:20] WARNING: C:/Users/Administrator/workspace/xgboost-win64_release_1.5.1/src/learner.cc:576: 
Parameters: { "logging_level" } might not be used.

  This could be a false alarm, with some parameters getting used by language bindings but
  then being mistakenly passed down to XGBoost core, or some parameter actually being used
  but getting flagged wrongly here. Please open an issue if you find any such cases.


[17:02:20] WARNING: C:/Users/Administrator/workspace/xgboost-win64_release_1.5.1/src/learner.cc:1115: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[LightGBM] [Warning] Unknown parameter: logging_level
[17:02:23] WARNING: C:/Users/Administrator/workspace/xgboost-win64_release_1.5.1/src/learner.cc:576: 
Parameters: { "logging_level" } might not be used.

  This could be a false alarm, with some parameters getting used by language bindings but
  then being mistakenly passed down to XGBoost core, or some parameter actually being used
  but getting flagged wrongly here. Please open an issue if you find any such cases.


[17:02:23] WARNING: C:/Users/Administrator/workspace/xgboost-win64_release_1.5.1/src/learner.cc:1115: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[LightGBM] [Warning] Unknown parameter: logging_level
[17:02:26] WARNING: C:/Users/Administrator/workspace/xgboost-win64_release_1.5.1/src/learner.cc:576: 
Parameters: { "logging_level" } might not be used.

  This could be a false alarm, with some parameters getting used by language bindings but
  then being mistakenly passed down to XGBoost core, or some parameter actually being used
  but getting flagged wrongly here. Please open an issue if you find any such cases.


[17:02:26] WARNING: C:/Users/Administrator/workspace/xgboost-win64_release_1.5.1/src/learner.cc:1115: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[LightGBM] [Warning] Unknown parameter: logging_level

reportDataframe_avg

	Model	Accuracy	R2	ROC_AUC	Precision_0	Precision_1	Recall_0	Recall_1	F1_Score	Combined
0	AdaBoostClassifier	0.844203	0.363101	0.895747	0.813062	0.869904	0.836951	0.852486	0.860289	2.103051
1	CatBoostClassifier	0.871014	0.479102	0.928439	0.886278	0.859523	0.821965	0.912277	0.884981	2.278555
2	DecisionTreeClassifier	0.778261	0.104285	0.775538	0.761031	0.791487	0.750958	0.800118	0.795749	1.658084
3	GradientBoostingClassifier	0.864493	0.454700	0.913440	0.869289	0.860686	0.839215	0.888558	0.873890	2.232632
4	KNeighborsClassifier	0.844928	0.372714	0.893223	0.856162	0.837617	0.788858	0.890710	0.863131	2.110864
5	LGBMClassifier	0.850725	0.393741	0.919019	0.868074	0.839663	0.782238	0.904987	0.870794	2.163485
6	LogisticRegression	0.857246	0.421611	0.918213	0.856110	0.858444	0.818133	0.888800	0.873150	2.197071
7	RandomForestClassifier	0.875362	0.494111	0.929173	0.886300	0.868222	0.824404	0.915136	0.890899	2.298647
8	SVC	0.850725	0.398919	0.909574	0.859239	0.845717	0.812591	0.884547	0.864025	2.159218
9	XGBClassifier	0.852899	0.403050	0.919731	0.852965	0.852273	0.810437	0.887019	0.869151	2.175679

Catboost Hyperparams

X = newDf.drop('HeartDisease', axis=1)
y= newDf['HeartDisease']

def objective(trial):
    # Define the hyperparameter search space
    params = {
        "iterations": trial.suggest_int("iterations", 100, 1000),
        "learning_rate": trial.suggest_loguniform("learning_rate", 0.01, 0.3),
        "depth": trial.suggest_int("depth", 1, 10),
        "l2_leaf_reg": trial.suggest_loguniform("l2_leaf_reg", 1e-9, 100),
    }
    
    # Split the data into train and validation sets
    X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=42)

    # Create and train the CatBoostClassifier with the suggested hyperparameters
    model = CatBoostClassifier(**params, logging_level='Silent')
    model.fit(X_train, y_train, eval_set=(X_val, y_val), early_stopping_rounds=50)

    # Calculate the evaluation metric to optimize (e.g., accuracy)
    accuracy = model.score(X_val, y_val)

    return accuracy

# Create a study object and optimize the objective function
study = optuna.create_study(direction="maximize")  # We want to maximize the accuracy
study.optimize(objective, n_trials=100)

# Get the best hyperparameters and their corresponding score
best_params = study.best_params
best_score = study.best_value

print("Best Hyperparameters:", best_params)
print("Best Score (Accuracy):", best_score)

[I 2023-07-31 17:02:27,104] A new study created in memory with name: no-name-abebc860-8a15-4d8d-96ad-e888367936d4
[I 2023-07-31 17:02:27,657] Trial 0 finished with value: 0.8695652173913043 and parameters: {'iterations': 720, 'learning_rate': 0.050381864384337456, 'depth': 8, 'l2_leaf_reg': 5.151473047597851e-07}. Best is trial 0 with value: 0.8695652173913043.
[I 2023-07-31 17:02:27,991] Trial 1 finished with value: 0.8804347826086957 and parameters: {'iterations': 110, 'learning_rate': 0.03617318272961226, 'depth': 5, 'l2_leaf_reg': 0.0007484340383610256}. Best is trial 1 with value: 0.8804347826086957.
[I 2023-07-31 17:02:28,615] Trial 2 finished with value: 0.8695652173913043 and parameters: {'iterations': 965, 'learning_rate': 0.037167782087731735, 'depth': 1, 'l2_leaf_reg': 0.08798915824701616}. Best is trial 1 with value: 0.8804347826086957.
[I 2023-07-31 17:02:28,932] Trial 3 finished with value: 0.8532608695652174 and parameters: {'iterations': 773, 'learning_rate': 0.2502008442552998, 'depth': 4, 'l2_leaf_reg': 0.0007137869576294317}. Best is trial 1 with value: 0.8804347826086957.
[I 2023-07-31 17:02:29,299] Trial 4 finished with value: 0.8804347826086957 and parameters: {'iterations': 416, 'learning_rate': 0.14183007124859565, 'depth': 3, 'l2_leaf_reg': 0.6047562568765807}. Best is trial 1 with value: 0.8804347826086957.
[I 2023-07-31 17:02:30,690] Trial 5 finished with value: 0.8532608695652174 and parameters: {'iterations': 377, 'learning_rate': 0.09545565729388585, 'depth': 10, 'l2_leaf_reg': 1.9658234904549985}. Best is trial 1 with value: 0.8804347826086957.
[I 2023-07-31 17:02:31,919] Trial 6 finished with value: 0.8804347826086957 and parameters: {'iterations': 812, 'learning_rate': 0.024446611806116087, 'depth': 10, 'l2_leaf_reg': 2.354108322999452e-09}. Best is trial 1 with value: 0.8804347826086957.
[I 2023-07-31 17:02:32,758] Trial 7 finished with value: 0.8478260869565217 and parameters: {'iterations': 415, 'learning_rate': 0.2093015578985169, 'depth': 10, 'l2_leaf_reg': 1.6551430557340853e-05}. Best is trial 1 with value: 0.8804347826086957.
[I 2023-07-31 17:02:33,192] Trial 8 finished with value: 0.8586956521739131 and parameters: {'iterations': 633, 'learning_rate': 0.04954537403490409, 'depth': 5, 'l2_leaf_reg': 0.001169520097757618}. Best is trial 1 with value: 0.8804347826086957.
[I 2023-07-31 17:02:33,826] Trial 9 finished with value: 0.8695652173913043 and parameters: {'iterations': 821, 'learning_rate': 0.014413248573597574, 'depth': 4, 'l2_leaf_reg': 1.989396359806507e-05}. Best is trial 1 with value: 0.8804347826086957.
[I 2023-07-31 17:02:34,376] Trial 10 finished with value: 0.8097826086956522 and parameters: {'iterations': 114, 'learning_rate': 0.010284110030639435, 'depth': 7, 'l2_leaf_reg': 70.69087678688591}. Best is trial 1 with value: 0.8804347826086957.
[I 2023-07-31 17:02:34,850] Trial 11 finished with value: 0.8804347826086957 and parameters: {'iterations': 111, 'learning_rate': 0.10024326571467933, 'depth': 2, 'l2_leaf_reg': 0.060867670593894634}. Best is trial 1 with value: 0.8804347826086957.
[I 2023-07-31 17:02:35,333] Trial 12 finished with value: 0.8804347826086957 and parameters: {'iterations': 297, 'learning_rate': 0.0937193560833953, 'depth': 3, 'l2_leaf_reg': 0.042057532866867864}. Best is trial 1 with value: 0.8804347826086957.
[I 2023-07-31 17:02:36,153] Trial 13 finished with value: 0.875 and parameters: {'iterations': 252, 'learning_rate': 0.025874758112120397, 'depth': 6, 'l2_leaf_reg': 60.407811936359444}. Best is trial 1 with value: 0.8804347826086957.
[I 2023-07-31 17:02:36,632] Trial 14 finished with value: 0.8858695652173914 and parameters: {'iterations': 534, 'learning_rate': 0.12173143764258439, 'depth': 3, 'l2_leaf_reg': 1.1218630866415618}. Best is trial 14 with value: 0.8858695652173914.
[I 2023-07-31 17:02:37,041] Trial 15 finished with value: 0.8695652173913043 and parameters: {'iterations': 602, 'learning_rate': 0.06355224205329012, 'depth': 5, 'l2_leaf_reg': 0.0025706998046741954}. Best is trial 14 with value: 0.8858695652173914.
[I 2023-07-31 17:02:37,407] Trial 16 finished with value: 0.8695652173913043 and parameters: {'iterations': 501, 'learning_rate': 0.167428025249983, 'depth': 1, 'l2_leaf_reg': 3.4703745649555042}. Best is trial 14 with value: 0.8858695652173914.
[I 2023-07-31 17:02:37,875] Trial 17 finished with value: 0.8369565217391305 and parameters: {'iterations': 230, 'learning_rate': 0.2775399783405939, 'depth': 7, 'l2_leaf_reg': 0.012410964461532522}. Best is trial 14 with value: 0.8858695652173914.
[I 2023-07-31 17:02:38,290] Trial 18 finished with value: 0.8913043478260869 and parameters: {'iterations': 967, 'learning_rate': 0.07427830244092323, 'depth': 3, 'l2_leaf_reg': 0.8688202452346111}. Best is trial 18 with value: 0.8913043478260869.
[I 2023-07-31 17:02:38,683] Trial 19 finished with value: 0.8858695652173914 and parameters: {'iterations': 993, 'learning_rate': 0.1429821749489193, 'depth': 3, 'l2_leaf_reg': 5.930838234148949}. Best is trial 18 with value: 0.8913043478260869.
[I 2023-07-31 17:02:39,150] Trial 20 finished with value: 0.8695652173913043 and parameters: {'iterations': 515, 'learning_rate': 0.07578469640513263, 'depth': 4, 'l2_leaf_reg': 0.4050728081262723}. Best is trial 18 with value: 0.8913043478260869.
[I 2023-07-31 17:02:39,532] Trial 21 finished with value: 0.875 and parameters: {'iterations': 986, 'learning_rate': 0.1158284001477013, 'depth': 2, 'l2_leaf_reg': 13.943759004413947}. Best is trial 18 with value: 0.8913043478260869.
[I 2023-07-31 17:02:39,932] Trial 22 finished with value: 0.8804347826086957 and parameters: {'iterations': 934, 'learning_rate': 0.15371794521427318, 'depth': 3, 'l2_leaf_reg': 6.07683268503343}. Best is trial 18 with value: 0.8913043478260869.
[I 2023-07-31 17:02:40,357] Trial 23 finished with value: 0.8804347826086957 and parameters: {'iterations': 877, 'learning_rate': 0.1260235811283461, 'depth': 2, 'l2_leaf_reg': 0.43777825390752234}. Best is trial 18 with value: 0.8913043478260869.
[I 2023-07-31 17:02:40,924] Trial 24 finished with value: 0.8913043478260869 and parameters: {'iterations': 713, 'learning_rate': 0.18582319405312353, 'depth': 3, 'l2_leaf_reg': 10.79238542072001}. Best is trial 18 with value: 0.8913043478260869.
[I 2023-07-31 17:02:41,634] Trial 25 finished with value: 0.875 and parameters: {'iterations': 678, 'learning_rate': 0.19291144684880132, 'depth': 2, 'l2_leaf_reg': 67.67960525343503}. Best is trial 18 with value: 0.8913043478260869.
[I 2023-07-31 17:02:41,999] Trial 26 finished with value: 0.8695652173913043 and parameters: {'iterations': 575, 'learning_rate': 0.2085567229758612, 'depth': 4, 'l2_leaf_reg': 0.1916747029142371}. Best is trial 18 with value: 0.8913043478260869.
[I 2023-07-31 17:02:42,482] Trial 27 finished with value: 0.875 and parameters: {'iterations': 719, 'learning_rate': 0.08233502457839208, 'depth': 3, 'l2_leaf_reg': 1.892659490009807}. Best is trial 18 with value: 0.8913043478260869.
[I 2023-07-31 17:02:42,957] Trial 28 finished with value: 0.8695652173913043 and parameters: {'iterations': 878, 'learning_rate': 0.12319113053091636, 'depth': 1, 'l2_leaf_reg': 16.218551591901566}. Best is trial 18 with value: 0.8913043478260869.
[I 2023-07-31 17:02:43,424] Trial 29 finished with value: 0.8586956521739131 and parameters: {'iterations': 701, 'learning_rate': 0.2997007806161126, 'depth': 6, 'l2_leaf_reg': 0.01404928853193876}. Best is trial 18 with value: 0.8913043478260869.
[I 2023-07-31 17:02:44,075] Trial 30 finished with value: 0.8641304347826086 and parameters: {'iterations': 487, 'learning_rate': 0.0685865847289507, 'depth': 8, 'l2_leaf_reg': 0.25307904581626883}. Best is trial 18 with value: 0.8913043478260869.
[I 2023-07-31 17:02:44,440] Trial 31 finished with value: 0.8804347826086957 and parameters: {'iterations': 904, 'learning_rate': 0.16479163204263067, 'depth': 3, 'l2_leaf_reg': 1.7914159006802375}. Best is trial 18 with value: 0.8913043478260869.
[I 2023-07-31 17:02:44,909] Trial 32 finished with value: 0.8858695652173914 and parameters: {'iterations': 774, 'learning_rate': 0.11408024141781611, 'depth': 2, 'l2_leaf_reg': 12.601294332324153}. Best is trial 18 with value: 0.8913043478260869.
[I 2023-07-31 17:02:45,315] Trial 33 finished with value: 0.8913043478260869 and parameters: {'iterations': 968, 'learning_rate': 0.1421276178148207, 'depth': 4, 'l2_leaf_reg': 1.0112584477856226}. Best is trial 18 with value: 0.8913043478260869.
[I 2023-07-31 17:02:45,697] Trial 34 finished with value: 0.8695652173913043 and parameters: {'iterations': 634, 'learning_rate': 0.23177833324150218, 'depth': 4, 'l2_leaf_reg': 0.8585371374393903}. Best is trial 18 with value: 0.8913043478260869.
[I 2023-07-31 17:02:46,099] Trial 35 finished with value: 0.8641304347826086 and parameters: {'iterations': 805, 'learning_rate': 0.18359086258607013, 'depth': 5, 'l2_leaf_reg': 0.11793175825303852}. Best is trial 18 with value: 0.8913043478260869.
[I 2023-07-31 17:02:46,566] Trial 36 finished with value: 0.8967391304347826 and parameters: {'iterations': 948, 'learning_rate': 0.08571922524005526, 'depth': 4, 'l2_leaf_reg': 1.1794757772097515}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:02:47,050] Trial 37 finished with value: 0.875 and parameters: {'iterations': 925, 'learning_rate': 0.09028172031536459, 'depth': 5, 'l2_leaf_reg': 25.59237154005505}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:02:47,701] Trial 38 finished with value: 0.875 and parameters: {'iterations': 858, 'learning_rate': 0.055230082301699186, 'depth': 4, 'l2_leaf_reg': 4.391014296416849}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:02:48,166] Trial 39 finished with value: 0.8478260869565217 and parameters: {'iterations': 952, 'learning_rate': 0.14626290759326055, 'depth': 6, 'l2_leaf_reg': 0.029381734416077752}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:02:48,617] Trial 40 finished with value: 0.8641304347826086 and parameters: {'iterations': 747, 'learning_rate': 0.08191903810600422, 'depth': 4, 'l2_leaf_reg': 0.13060130733498798}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:02:49,075] Trial 41 finished with value: 0.8804347826086957 and parameters: {'iterations': 843, 'learning_rate': 0.10359497705055605, 'depth': 3, 'l2_leaf_reg': 0.6124268727079343}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:02:49,582] Trial 42 finished with value: 0.8804347826086957 and parameters: {'iterations': 943, 'learning_rate': 0.13506645914979368, 'depth': 4, 'l2_leaf_reg': 1.2045450790934749}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:02:50,265] Trial 43 finished with value: 0.8913043478260869 and parameters: {'iterations': 1000, 'learning_rate': 0.10340879985980185, 'depth': 5, 'l2_leaf_reg': 1.6965621141671228}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:02:50,832] Trial 44 finished with value: 0.8641304347826086 and parameters: {'iterations': 988, 'learning_rate': 0.1024018658068596, 'depth': 5, 'l2_leaf_reg': 94.90537728272331}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:02:51,349] Trial 45 finished with value: 0.875 and parameters: {'iterations': 903, 'learning_rate': 0.07553747304583262, 'depth': 5, 'l2_leaf_reg': 5.777070470119603}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:02:51,715] Trial 46 finished with value: 0.8695652173913043 and parameters: {'iterations': 999, 'learning_rate': 0.23423713373140104, 'depth': 6, 'l2_leaf_reg': 0.08116877208123857}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:02:52,400] Trial 47 finished with value: 0.8858695652173914 and parameters: {'iterations': 778, 'learning_rate': 0.1716538671974615, 'depth': 7, 'l2_leaf_reg': 27.134745590039664}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:02:52,965] Trial 48 finished with value: 0.8804347826086957 and parameters: {'iterations': 827, 'learning_rate': 0.05995545269274146, 'depth': 4, 'l2_leaf_reg': 2.5570419427468694}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:02:53,566] Trial 49 finished with value: 0.8913043478260869 and parameters: {'iterations': 904, 'learning_rate': 0.04888501057749291, 'depth': 3, 'l2_leaf_reg': 0.3748424802157774}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:02:53,964] Trial 50 finished with value: 0.8641304347826086 and parameters: {'iterations': 961, 'learning_rate': 0.10854298989206887, 'depth': 5, 'l2_leaf_reg': 0.003435414062991254}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:02:54,509] Trial 51 finished with value: 0.8804347826086957 and parameters: {'iterations': 905, 'learning_rate': 0.04768865865797106, 'depth': 2, 'l2_leaf_reg': 0.339202093154479}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:02:54,983] Trial 52 finished with value: 0.8804347826086957 and parameters: {'iterations': 948, 'learning_rate': 0.09024637380835622, 'depth': 3, 'l2_leaf_reg': 1.1566311843897192}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:02:55,431] Trial 53 finished with value: 0.875 and parameters: {'iterations': 860, 'learning_rate': 0.048750108603786285, 'depth': 4, 'l2_leaf_reg': 7.5119707378343294}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:02:55,900] Trial 54 finished with value: 0.8858695652173914 and parameters: {'iterations': 904, 'learning_rate': 0.06500095625505824, 'depth': 3, 'l2_leaf_reg': 2.4451060668930764}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:02:56,458] Trial 55 finished with value: 0.8804347826086957 and parameters: {'iterations': 970, 'learning_rate': 0.043050023460006005, 'depth': 4, 'l2_leaf_reg': 0.5549419806994629}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:02:56,831] Trial 56 finished with value: 0.8641304347826086 and parameters: {'iterations': 923, 'learning_rate': 0.13332298015255745, 'depth': 3, 'l2_leaf_reg': 0.22597180806766298}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:02:57,415] Trial 57 finished with value: 0.8695652173913043 and parameters: {'iterations': 802, 'learning_rate': 0.07126608380032318, 'depth': 1, 'l2_leaf_reg': 31.178077093472634}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:02:57,862] Trial 58 finished with value: 0.8858695652173914 and parameters: {'iterations': 883, 'learning_rate': 0.059093892158079406, 'depth': 2, 'l2_leaf_reg': 0.043821357301471416}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:02:58,928] Trial 59 finished with value: 0.8913043478260869 and parameters: {'iterations': 969, 'learning_rate': 0.09423316185286316, 'depth': 9, 'l2_leaf_reg': 9.728327772374502}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:02:59,365] Trial 60 finished with value: 0.8695652173913043 and parameters: {'iterations': 674, 'learning_rate': 0.1532152777460022, 'depth': 5, 'l2_leaf_reg': 3.7232336284225744}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:03:00,584] Trial 61 finished with value: 0.875 and parameters: {'iterations': 959, 'learning_rate': 0.09335970974782513, 'depth': 9, 'l2_leaf_reg': 11.672233322334005}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:03:01,400] Trial 62 finished with value: 0.8858695652173914 and parameters: {'iterations': 996, 'learning_rate': 0.08275274821270846, 'depth': 9, 'l2_leaf_reg': 0.9178349420129056}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:03:03,277] Trial 63 finished with value: 0.8804347826086957 and parameters: {'iterations': 963, 'learning_rate': 0.10940757227689599, 'depth': 10, 'l2_leaf_reg': 32.43534414086198}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:03:03,649] Trial 64 finished with value: 0.8858695652173914 and parameters: {'iterations': 923, 'learning_rate': 0.12803781426221583, 'depth': 3, 'l2_leaf_reg': 7.614617210892237}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:03:04,149] Trial 65 finished with value: 0.8695652173913043 and parameters: {'iterations': 843, 'learning_rate': 0.06938702826254287, 'depth': 4, 'l2_leaf_reg': 2.566032542430117}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:03:04,675] Trial 66 finished with value: 0.8586956521739131 and parameters: {'iterations': 879, 'learning_rate': 0.07899556937946309, 'depth': 7, 'l2_leaf_reg': 0.5645877896374343}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:03:05,383] Trial 67 finished with value: 0.8586956521739131 and parameters: {'iterations': 472, 'learning_rate': 0.11622368731266561, 'depth': 9, 'l2_leaf_reg': 0.21228589875368217}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:03:05,849] Trial 68 finished with value: 0.8804347826086957 and parameters: {'iterations': 971, 'learning_rate': 0.09396339275713758, 'depth': 3, 'l2_leaf_reg': 1.6016955298951792}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:03:06,224] Trial 69 finished with value: 0.8641304347826086 and parameters: {'iterations': 930, 'learning_rate': 0.14347165278254825, 'depth': 2, 'l2_leaf_reg': 14.441473246246849}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:03:06,657] Trial 70 finished with value: 0.8804347826086957 and parameters: {'iterations': 877, 'learning_rate': 0.10270498353337358, 'depth': 4, 'l2_leaf_reg': 46.28593128687329}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:03:07,116] Trial 71 finished with value: 0.8913043478260869 and parameters: {'iterations': 609, 'learning_rate': 0.12348676287188651, 'depth': 3, 'l2_leaf_reg': 4.297064540931493}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:03:07,525] Trial 72 finished with value: 0.8858695652173914 and parameters: {'iterations': 635, 'learning_rate': 0.1218569053916226, 'depth': 3, 'l2_leaf_reg': 4.110182790129383}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:03:07,983] Trial 73 finished with value: 0.8695652173913043 and parameters: {'iterations': 563, 'learning_rate': 0.08734673476970886, 'depth': 2, 'l2_leaf_reg': 13.234228627570866}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:03:08,416] Trial 74 finished with value: 0.8858695652173914 and parameters: {'iterations': 733, 'learning_rate': 0.1601337884759598, 'depth': 3, 'l2_leaf_reg': 98.75626688857913}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:03:09,059] Trial 75 finished with value: 0.8641304347826086 and parameters: {'iterations': 382, 'learning_rate': 0.0969872253618672, 'depth': 8, 'l2_leaf_reg': 1.1130917051680325}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:03:09,448] Trial 76 finished with value: 0.8858695652173914 and parameters: {'iterations': 595, 'learning_rate': 0.1932466111079506, 'depth': 4, 'l2_leaf_reg': 0.38104812931947085}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:03:09,941] Trial 77 finished with value: 0.8913043478260869 and parameters: {'iterations': 975, 'learning_rate': 0.13243916399447808, 'depth': 5, 'l2_leaf_reg': 6.991274116765668}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:03:10,457] Trial 78 finished with value: 0.8804347826086957 and parameters: {'iterations': 667, 'learning_rate': 0.07417129531660105, 'depth': 4, 'l2_leaf_reg': 0.12506795046647565}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:03:11,049] Trial 79 finished with value: 0.8586956521739131 and parameters: {'iterations': 152, 'learning_rate': 0.11565388933732236, 'depth': 3, 'l2_leaf_reg': 1.9199321313811486}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:03:11,709] Trial 80 finished with value: 0.8913043478260869 and parameters: {'iterations': 938, 'learning_rate': 0.08301878971654222, 'depth': 3, 'l2_leaf_reg': 0.7038669785754722}. Best is trial 36 with value: 0.8967391304347826.
[I 2023-07-31 17:03:12,132] Trial 81 finished with value: 0.9021739130434783 and parameters: {'iterations': 993, 'learning_rate': 0.1360622710958461, 'depth': 5, 'l2_leaf_reg': 6.605911256918625}. Best is trial 81 with value: 0.9021739130434783.
[I 2023-07-31 17:03:12,599] Trial 82 finished with value: 0.875 and parameters: {'iterations': 994, 'learning_rate': 0.14703526139738243, 'depth': 6, 'l2_leaf_reg': 3.5554198999571374}. Best is trial 81 with value: 0.9021739130434783.
[I 2023-07-31 17:03:13,132] Trial 83 finished with value: 0.907608695652174 and parameters: {'iterations': 909, 'learning_rate': 0.10795847719199778, 'depth': 5, 'l2_leaf_reg': 17.512176033781234}. Best is trial 83 with value: 0.907608695652174.
[I 2023-07-31 17:03:13,658] Trial 84 finished with value: 0.8913043478260869 and parameters: {'iterations': 907, 'learning_rate': 0.10200397008354779, 'depth': 5, 'l2_leaf_reg': 56.22689660961723}. Best is trial 83 with value: 0.907608695652174.
[I 2023-07-31 17:03:14,081] Trial 85 finished with value: 0.8858695652173914 and parameters: {'iterations': 946, 'learning_rate': 0.17196873387141953, 'depth': 6, 'l2_leaf_reg': 23.89254521086023}. Best is trial 83 with value: 0.907608695652174.
[I 2023-07-31 17:03:14,574] Trial 86 finished with value: 0.8967391304347826 and parameters: {'iterations': 980, 'learning_rate': 0.11192128894408956, 'depth': 5, 'l2_leaf_reg': 8.278519853917414}. Best is trial 83 with value: 0.907608695652174.
[I 2023-07-31 17:03:15,065] Trial 87 finished with value: 0.875 and parameters: {'iterations': 856, 'learning_rate': 0.13830183201382548, 'depth': 5, 'l2_leaf_reg': 17.493667913952578}. Best is trial 83 with value: 0.907608695652174.
[I 2023-07-31 17:03:15,456] Trial 88 finished with value: 0.8804347826086957 and parameters: {'iterations': 1000, 'learning_rate': 0.10683363219695703, 'depth': 5, 'l2_leaf_reg': 1.4565370578132621}. Best is trial 83 with value: 0.907608695652174.
[I 2023-07-31 17:03:15,974] Trial 89 finished with value: 0.8804347826086957 and parameters: {'iterations': 893, 'learning_rate': 0.1571627183631364, 'depth': 6, 'l2_leaf_reg': 0.3286758441762458}. Best is trial 83 with value: 0.907608695652174.
[I 2023-07-31 17:03:16,499] Trial 90 finished with value: 0.8804347826086957 and parameters: {'iterations': 917, 'learning_rate': 0.11287770740622029, 'depth': 5, 'l2_leaf_reg': 44.72714161156781}. Best is trial 83 with value: 0.907608695652174.
[I 2023-07-31 17:03:16,957] Trial 91 finished with value: 0.8858695652173914 and parameters: {'iterations': 979, 'learning_rate': 0.08798029473651599, 'depth': 4, 'l2_leaf_reg': 11.029682660358176}. Best is trial 83 with value: 0.907608695652174.
[I 2023-07-31 17:03:17,432] Trial 92 finished with value: 0.8858695652173914 and parameters: {'iterations': 947, 'learning_rate': 0.09438268707898557, 'depth': 5, 'l2_leaf_reg': 8.024781833084779}. Best is trial 83 with value: 0.907608695652174.
[I 2023-07-31 17:03:17,965] Trial 93 finished with value: 0.8913043478260869 and parameters: {'iterations': 972, 'learning_rate': 0.07676498490956776, 'depth': 6, 'l2_leaf_reg': 2.3442536449882745}. Best is trial 83 with value: 0.907608695652174.
[I 2023-07-31 17:03:18,383] Trial 94 finished with value: 0.875 and parameters: {'iterations': 934, 'learning_rate': 0.1303227056743962, 'depth': 4, 'l2_leaf_reg': 19.733282643787774}. Best is trial 83 with value: 0.907608695652174.
[I 2023-07-31 17:03:18,891] Trial 95 finished with value: 0.8967391304347826 and parameters: {'iterations': 827, 'learning_rate': 0.10156112258880623, 'depth': 5, 'l2_leaf_reg': 5.914929117741083}. Best is trial 83 with value: 0.907608695652174.
[I 2023-07-31 17:03:19,390] Trial 96 finished with value: 0.875 and parameters: {'iterations': 823, 'learning_rate': 0.11645898965230396, 'depth': 5, 'l2_leaf_reg': 0.9041390784780495}. Best is trial 83 with value: 0.907608695652174.
[I 2023-07-31 17:03:20,090] Trial 97 finished with value: 0.8913043478260869 and parameters: {'iterations': 864, 'learning_rate': 0.10063801319750412, 'depth': 6, 'l2_leaf_reg': 5.931838853480844}. Best is trial 83 with value: 0.907608695652174.
[I 2023-07-31 17:03:20,676] Trial 98 finished with value: 0.8858695652173914 and parameters: {'iterations': 803, 'learning_rate': 0.06633498859337436, 'depth': 4, 'l2_leaf_reg': 3.8495433098228706}. Best is trial 83 with value: 0.907608695652174.
[I 2023-07-31 17:03:21,441] Trial 99 finished with value: 0.8478260869565217 and parameters: {'iterations': 891, 'learning_rate': 0.13941933213851848, 'depth': 5, 'l2_leaf_reg': 0.5455303153380975}. Best is trial 83 with value: 0.907608695652174.

Best Hyperparameters: {'iterations': 909, 'learning_rate': 0.10795847719199778, 'depth': 5, 'l2_leaf_reg': 17.512176033781234}
Best Score (Accuracy): 0.907608695652174

RandomForest Hyperparams

X = newDf.drop('HeartDisease', axis=1)
y = newDf['HeartDisease']

def objective(trial):
    # Define the hyperparameter search space
    params = {
        "n_estimators": trial.suggest_int("n_estimators", 100, 1000),
        "max_depth": trial.suggest_int("max_depth", 1, 32),
        "min_samples_split": trial.suggest_int("min_samples_split", 2, 20),
        "min_samples_leaf": trial.suggest_int("min_samples_leaf", 1, 20),
        "max_features": trial.suggest_categorical("max_features", ["auto", "sqrt", "log2"]),
    }
    
    # Split the data into train and validation sets
    X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=42)

    # Create and train the RandomForestClassifier with the suggested hyperparameters
    model = RandomForestClassifier(**params)
    model.fit(X_train, y_train)

    # Calculate the evaluation metric to optimize (e.g., accuracy)
    accuracy = model.score(X_val, y_val)

    return accuracy

# Create a study object and optimize the objective function
study = optuna.create_study(direction="maximize")  # We want to maximize the accuracy
study.optimize(objective, n_trials=100)

# Get the best hyperparameters and their corresponding score
best_params = study.best_params
best_score = study.best_value

print("Best Hyperparameters:", best_params)
print("Best Score (Accuracy):", best_score)

[I 2023-07-31 17:03:21,459] A new study created in memory with name: no-name-29b35ff8-363c-477e-833e-87d49712e778
[I 2023-07-31 17:03:21,810] Trial 0 finished with value: 0.8586956521739131 and parameters: {'n_estimators': 274, 'max_depth': 24, 'min_samples_split': 7, 'min_samples_leaf': 9, 'max_features': 'auto'}. Best is trial 0 with value: 0.8586956521739131.
[I 2023-07-31 17:03:22,618] Trial 1 finished with value: 0.8478260869565217 and parameters: {'n_estimators': 800, 'max_depth': 15, 'min_samples_split': 3, 'min_samples_leaf': 18, 'max_features': 'auto'}. Best is trial 0 with value: 0.8586956521739131.
[I 2023-07-31 17:03:23,560] Trial 2 finished with value: 0.8695652173913043 and parameters: {'n_estimators': 822, 'max_depth': 32, 'min_samples_split': 10, 'min_samples_leaf': 5, 'max_features': 'log2'}. Best is trial 2 with value: 0.8695652173913043.
[I 2023-07-31 17:03:24,216] Trial 3 finished with value: 0.8369565217391305 and parameters: {'n_estimators': 684, 'max_depth': 2, 'min_samples_split': 6, 'min_samples_leaf': 10, 'max_features': 'sqrt'}. Best is trial 2 with value: 0.8695652173913043.
[I 2023-07-31 17:03:24,643] Trial 4 finished with value: 0.8532608695652174 and parameters: {'n_estimators': 389, 'max_depth': 31, 'min_samples_split': 12, 'min_samples_leaf': 15, 'max_features': 'log2'}. Best is trial 2 with value: 0.8695652173913043.
[I 2023-07-31 17:03:25,525] Trial 5 finished with value: 0.8586956521739131 and parameters: {'n_estimators': 874, 'max_depth': 20, 'min_samples_split': 7, 'min_samples_leaf': 13, 'max_features': 'sqrt'}. Best is trial 2 with value: 0.8695652173913043.
[I 2023-07-31 17:03:26,149] Trial 6 finished with value: 0.8369565217391305 and parameters: {'n_estimators': 675, 'max_depth': 2, 'min_samples_split': 15, 'min_samples_leaf': 9, 'max_features': 'log2'}. Best is trial 2 with value: 0.8695652173913043.
[I 2023-07-31 17:03:26,520] Trial 7 finished with value: 0.8804347826086957 and parameters: {'n_estimators': 331, 'max_depth': 9, 'min_samples_split': 11, 'min_samples_leaf': 4, 'max_features': 'sqrt'}. Best is trial 7 with value: 0.8804347826086957.
[I 2023-07-31 17:03:27,068] Trial 8 finished with value: 0.8532608695652174 and parameters: {'n_estimators': 561, 'max_depth': 4, 'min_samples_split': 17, 'min_samples_leaf': 6, 'max_features': 'sqrt'}. Best is trial 7 with value: 0.8804347826086957.
[I 2023-07-31 17:03:27,772] Trial 9 finished with value: 0.8641304347826086 and parameters: {'n_estimators': 667, 'max_depth': 26, 'min_samples_split': 5, 'min_samples_leaf': 6, 'max_features': 'log2'}. Best is trial 7 with value: 0.8804347826086957.
[I 2023-07-31 17:03:27,923] Trial 10 finished with value: 0.875 and parameters: {'n_estimators': 113, 'max_depth': 10, 'min_samples_split': 19, 'min_samples_leaf': 1, 'max_features': 'sqrt'}. Best is trial 7 with value: 0.8804347826086957.
[I 2023-07-31 17:03:28,074] Trial 11 finished with value: 0.8858695652173914 and parameters: {'n_estimators': 118, 'max_depth': 10, 'min_samples_split': 20, 'min_samples_leaf': 1, 'max_features': 'sqrt'}. Best is trial 11 with value: 0.8858695652173914.
[I 2023-07-31 17:03:28,205] Trial 12 finished with value: 0.8804347826086957 and parameters: {'n_estimators': 105, 'max_depth': 10, 'min_samples_split': 13, 'min_samples_leaf': 1, 'max_features': 'sqrt'}. Best is trial 11 with value: 0.8858695652173914.
[I 2023-07-31 17:03:28,553] Trial 13 finished with value: 0.8695652173913043 and parameters: {'n_estimators': 312, 'max_depth': 9, 'min_samples_split': 20, 'min_samples_leaf': 3, 'max_features': 'sqrt'}. Best is trial 11 with value: 0.8858695652173914.
[I 2023-07-31 17:03:28,826] Trial 14 finished with value: 0.8641304347826086 and parameters: {'n_estimators': 231, 'max_depth': 15, 'min_samples_split': 10, 'min_samples_leaf': 4, 'max_features': 'sqrt'}. Best is trial 11 with value: 0.8858695652173914.
[I 2023-07-31 17:03:29,294] Trial 15 finished with value: 0.8695652173913043 and parameters: {'n_estimators': 450, 'max_depth': 6, 'min_samples_split': 15, 'min_samples_leaf': 1, 'max_features': 'sqrt'}. Best is trial 11 with value: 0.8858695652173914.
[I 2023-07-31 17:03:29,518] Trial 16 finished with value: 0.8641304347826086 and parameters: {'n_estimators': 191, 'max_depth': 12, 'min_samples_split': 9, 'min_samples_leaf': 7, 'max_features': 'auto'}. Best is trial 11 with value: 0.8858695652173914.
[I 2023-07-31 17:03:29,987] Trial 17 finished with value: 0.875 and parameters: {'n_estimators': 402, 'max_depth': 20, 'min_samples_split': 14, 'min_samples_leaf': 3, 'max_features': 'sqrt'}. Best is trial 11 with value: 0.8858695652173914.
[I 2023-07-31 17:03:31,002] Trial 18 finished with value: 0.8586956521739131 and parameters: {'n_estimators': 992, 'max_depth': 6, 'min_samples_split': 17, 'min_samples_leaf': 13, 'max_features': 'sqrt'}. Best is trial 11 with value: 0.8858695652173914.
[I 2023-07-31 17:03:31,611] Trial 19 finished with value: 0.8586956521739131 and parameters: {'n_estimators': 525, 'max_depth': 18, 'min_samples_split': 17, 'min_samples_leaf': 8, 'max_features': 'auto'}. Best is trial 11 with value: 0.8858695652173914.
[I 2023-07-31 17:03:31,837] Trial 20 finished with value: 0.8695652173913043 and parameters: {'n_estimators': 194, 'max_depth': 13, 'min_samples_split': 2, 'min_samples_leaf': 3, 'max_features': 'sqrt'}. Best is trial 11 with value: 0.8858695652173914.
[I 2023-07-31 17:03:31,977] Trial 21 finished with value: 0.8858695652173914 and parameters: {'n_estimators': 108, 'max_depth': 8, 'min_samples_split': 12, 'min_samples_leaf': 1, 'max_features': 'sqrt'}. Best is trial 11 with value: 0.8858695652173914.
[I 2023-07-31 17:03:32,364] Trial 22 finished with value: 0.8804347826086957 and parameters: {'n_estimators': 348, 'max_depth': 7, 'min_samples_split': 12, 'min_samples_leaf': 1, 'max_features': 'sqrt'}. Best is trial 11 with value: 0.8858695652173914.
[I 2023-07-31 17:03:32,550] Trial 23 finished with value: 0.8695652173913043 and parameters: {'n_estimators': 156, 'max_depth': 12, 'min_samples_split': 9, 'min_samples_leaf': 4, 'max_features': 'sqrt'}. Best is trial 11 with value: 0.8858695652173914.
[I 2023-07-31 17:03:32,826] Trial 24 finished with value: 0.842391304347826 and parameters: {'n_estimators': 259, 'max_depth': 8, 'min_samples_split': 19, 'min_samples_leaf': 20, 'max_features': 'sqrt'}. Best is trial 11 with value: 0.8858695652173914.
[I 2023-07-31 17:03:33,006] Trial 25 finished with value: 0.8532608695652174 and parameters: {'n_estimators': 164, 'max_depth': 4, 'min_samples_split': 11, 'min_samples_leaf': 3, 'max_features': 'sqrt'}. Best is trial 11 with value: 0.8858695652173914.
[I 2023-07-31 17:03:33,513] Trial 26 finished with value: 0.875 and parameters: {'n_estimators': 469, 'max_depth': 14, 'min_samples_split': 15, 'min_samples_leaf': 2, 'max_features': 'sqrt'}. Best is trial 11 with value: 0.8858695652173914.
[I 2023-07-31 17:03:33,857] Trial 27 finished with value: 0.875 and parameters: {'n_estimators': 307, 'max_depth': 11, 'min_samples_split': 8, 'min_samples_leaf': 5, 'max_features': 'sqrt'}. Best is trial 11 with value: 0.8858695652173914.
[I 2023-07-31 17:03:33,987] Trial 28 finished with value: 0.8532608695652174 and parameters: {'n_estimators': 104, 'max_depth': 17, 'min_samples_split': 4, 'min_samples_leaf': 12, 'max_features': 'log2'}. Best is trial 11 with value: 0.8858695652173914.
[I 2023-07-31 17:03:34,252] Trial 29 finished with value: 0.8478260869565217 and parameters: {'n_estimators': 242, 'max_depth': 4, 'min_samples_split': 13, 'min_samples_leaf': 7, 'max_features': 'auto'}. Best is trial 11 with value: 0.8858695652173914.
[I 2023-07-31 17:03:34,537] Trial 30 finished with value: 0.8369565217391305 and parameters: {'n_estimators': 290, 'max_depth': 1, 'min_samples_split': 16, 'min_samples_leaf': 2, 'max_features': 'auto'}. Best is trial 11 with value: 0.8858695652173914.
[I 2023-07-31 17:03:34,695] Trial 31 finished with value: 0.8913043478260869 and parameters: {'n_estimators': 126, 'max_depth': 10, 'min_samples_split': 13, 'min_samples_leaf': 1, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:34,940] Trial 32 finished with value: 0.8804347826086957 and parameters: {'n_estimators': 202, 'max_depth': 8, 'min_samples_split': 12, 'min_samples_leaf': 2, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:35,108] Trial 33 finished with value: 0.8641304347826086 and parameters: {'n_estimators': 146, 'max_depth': 6, 'min_samples_split': 10, 'min_samples_leaf': 4, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:35,501] Trial 34 finished with value: 0.8586956521739131 and parameters: {'n_estimators': 349, 'max_depth': 10, 'min_samples_split': 11, 'min_samples_leaf': 5, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:35,772] Trial 35 finished with value: 0.8695652173913043 and parameters: {'n_estimators': 229, 'max_depth': 15, 'min_samples_split': 13, 'min_samples_leaf': 2, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:35,948] Trial 36 finished with value: 0.8532608695652174 and parameters: {'n_estimators': 159, 'max_depth': 13, 'min_samples_split': 7, 'min_samples_leaf': 17, 'max_features': 'log2'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:36,261] Trial 37 finished with value: 0.8858695652173914 and parameters: {'n_estimators': 272, 'max_depth': 8, 'min_samples_split': 14, 'min_samples_leaf': 1, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:36,533] Trial 38 finished with value: 0.842391304347826 and parameters: {'n_estimators': 257, 'max_depth': 3, 'min_samples_split': 14, 'min_samples_leaf': 1, 'max_features': 'auto'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:36,670] Trial 39 finished with value: 0.8641304347826086 and parameters: {'n_estimators': 114, 'max_depth': 24, 'min_samples_split': 18, 'min_samples_leaf': 10, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:36,896] Trial 40 finished with value: 0.8532608695652174 and parameters: {'n_estimators': 202, 'max_depth': 5, 'min_samples_split': 16, 'min_samples_leaf': 15, 'max_features': 'log2'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:37,301] Trial 41 finished with value: 0.8695652173913043 and parameters: {'n_estimators': 360, 'max_depth': 8, 'min_samples_split': 14, 'min_samples_leaf': 2, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:37,928] Trial 42 finished with value: 0.8695652173913043 and parameters: {'n_estimators': 583, 'max_depth': 11, 'min_samples_split': 12, 'min_samples_leaf': 5, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:38,106] Trial 43 finished with value: 0.8858695652173914 and parameters: {'n_estimators': 138, 'max_depth': 9, 'min_samples_split': 9, 'min_samples_leaf': 1, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:38,285] Trial 44 finished with value: 0.8913043478260869 and parameters: {'n_estimators': 143, 'max_depth': 9, 'min_samples_split': 9, 'min_samples_leaf': 1, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:39,086] Trial 45 finished with value: 0.8804347826086957 and parameters: {'n_estimators': 735, 'max_depth': 7, 'min_samples_split': 8, 'min_samples_leaf': 3, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:39,302] Trial 46 finished with value: 0.8858695652173914 and parameters: {'n_estimators': 181, 'max_depth': 11, 'min_samples_split': 7, 'min_samples_leaf': 1, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:39,450] Trial 47 finished with value: 0.875 and parameters: {'n_estimators': 117, 'max_depth': 16, 'min_samples_split': 6, 'min_samples_leaf': 2, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:39,762] Trial 48 finished with value: 0.8586956521739131 and parameters: {'n_estimators': 279, 'max_depth': 29, 'min_samples_split': 12, 'min_samples_leaf': 6, 'max_features': 'log2'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:40,016] Trial 49 finished with value: 0.8695652173913043 and parameters: {'n_estimators': 222, 'max_depth': 9, 'min_samples_split': 20, 'min_samples_leaf': 4, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:40,175] Trial 50 finished with value: 0.8315217391304348 and parameters: {'n_estimators': 150, 'max_depth': 2, 'min_samples_split': 10, 'min_samples_leaf': 3, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:40,337] Trial 51 finished with value: 0.8695652173913043 and parameters: {'n_estimators': 130, 'max_depth': 9, 'min_samples_split': 9, 'min_samples_leaf': 1, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:40,462] Trial 52 finished with value: 0.8586956521739131 and parameters: {'n_estimators': 102, 'max_depth': 5, 'min_samples_split': 11, 'min_samples_leaf': 1, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:40,665] Trial 53 finished with value: 0.8858695652173914 and parameters: {'n_estimators': 173, 'max_depth': 12, 'min_samples_split': 13, 'min_samples_leaf': 2, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:40,917] Trial 54 finished with value: 0.8804347826086957 and parameters: {'n_estimators': 216, 'max_depth': 7, 'min_samples_split': 8, 'min_samples_leaf': 1, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:41,096] Trial 55 finished with value: 0.875 and parameters: {'n_estimators': 147, 'max_depth': 14, 'min_samples_split': 9, 'min_samples_leaf': 3, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:41,544] Trial 56 finished with value: 0.8695652173913043 and parameters: {'n_estimators': 392, 'max_depth': 9, 'min_samples_split': 6, 'min_samples_leaf': 2, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:42,509] Trial 57 finished with value: 0.8804347826086957 and parameters: {'n_estimators': 912, 'max_depth': 11, 'min_samples_split': 10, 'min_samples_leaf': 4, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:42,730] Trial 58 finished with value: 0.875 and parameters: {'n_estimators': 182, 'max_depth': 10, 'min_samples_split': 16, 'min_samples_leaf': 1, 'max_features': 'auto'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:42,852] Trial 59 finished with value: 0.8695652173913043 and parameters: {'n_estimators': 100, 'max_depth': 6, 'min_samples_split': 15, 'min_samples_leaf': 3, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:43,163] Trial 60 finished with value: 0.875 and parameters: {'n_estimators': 273, 'max_depth': 19, 'min_samples_split': 8, 'min_samples_leaf': 2, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:43,380] Trial 61 finished with value: 0.8913043478260869 and parameters: {'n_estimators': 174, 'max_depth': 10, 'min_samples_split': 7, 'min_samples_leaf': 1, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:43,549] Trial 62 finished with value: 0.875 and parameters: {'n_estimators': 132, 'max_depth': 8, 'min_samples_split': 5, 'min_samples_leaf': 1, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:43,758] Trial 63 finished with value: 0.875 and parameters: {'n_estimators': 179, 'max_depth': 10, 'min_samples_split': 19, 'min_samples_leaf': 1, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:44,042] Trial 64 finished with value: 0.8804347826086957 and parameters: {'n_estimators': 234, 'max_depth': 12, 'min_samples_split': 9, 'min_samples_leaf': 2, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:44,211] Trial 65 finished with value: 0.875 and parameters: {'n_estimators': 136, 'max_depth': 13, 'min_samples_split': 11, 'min_samples_leaf': 3, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:44,561] Trial 66 finished with value: 0.875 and parameters: {'n_estimators': 319, 'max_depth': 7, 'min_samples_split': 7, 'min_samples_leaf': 4, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:44,789] Trial 67 finished with value: 0.875 and parameters: {'n_estimators': 203, 'max_depth': 5, 'min_samples_split': 10, 'min_samples_leaf': 1, 'max_features': 'log2'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:44,983] Trial 68 finished with value: 0.8695652173913043 and parameters: {'n_estimators': 163, 'max_depth': 9, 'min_samples_split': 13, 'min_samples_leaf': 2, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:45,277] Trial 69 finished with value: 0.875 and parameters: {'n_estimators': 250, 'max_depth': 14, 'min_samples_split': 14, 'min_samples_leaf': 5, 'max_features': 'auto'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:45,438] Trial 70 finished with value: 0.875 and parameters: {'n_estimators': 128, 'max_depth': 22, 'min_samples_split': 4, 'min_samples_leaf': 3, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:45,669] Trial 71 finished with value: 0.8804347826086957 and parameters: {'n_estimators': 189, 'max_depth': 11, 'min_samples_split': 7, 'min_samples_leaf': 1, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:45,879] Trial 72 finished with value: 0.8858695652173914 and parameters: {'n_estimators': 172, 'max_depth': 10, 'min_samples_split': 6, 'min_samples_leaf': 1, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:46,101] Trial 73 finished with value: 0.8804347826086957 and parameters: {'n_estimators': 142, 'max_depth': 8, 'min_samples_split': 7, 'min_samples_leaf': 2, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:46,356] Trial 74 finished with value: 0.8804347826086957 and parameters: {'n_estimators': 212, 'max_depth': 13, 'min_samples_split': 5, 'min_samples_leaf': 1, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:47,054] Trial 75 finished with value: 0.8804347826086957 and parameters: {'n_estimators': 622, 'max_depth': 11, 'min_samples_split': 8, 'min_samples_leaf': 3, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:47,217] Trial 76 finished with value: 0.8641304347826086 and parameters: {'n_estimators': 122, 'max_depth': 6, 'min_samples_split': 11, 'min_samples_leaf': 2, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:47,743] Trial 77 finished with value: 0.8586956521739131 and parameters: {'n_estimators': 473, 'max_depth': 9, 'min_samples_split': 18, 'min_samples_leaf': 12, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:47,941] Trial 78 finished with value: 0.8804347826086957 and parameters: {'n_estimators': 163, 'max_depth': 8, 'min_samples_split': 9, 'min_samples_leaf': 1, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:48,296] Trial 79 finished with value: 0.8804347826086957 and parameters: {'n_estimators': 294, 'max_depth': 10, 'min_samples_split': 6, 'min_samples_leaf': 2, 'max_features': 'log2'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:48,567] Trial 80 finished with value: 0.8586956521739131 and parameters: {'n_estimators': 246, 'max_depth': 7, 'min_samples_split': 12, 'min_samples_leaf': 9, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:48,791] Trial 81 finished with value: 0.8804347826086957 and parameters: {'n_estimators': 181, 'max_depth': 12, 'min_samples_split': 13, 'min_samples_leaf': 2, 'max_features': 'sqrt'}. Best is trial 31 with value: 0.8913043478260869.
[I 2023-07-31 17:03:48,996] Trial 82 finished with value: 0.8967391304347826 and parameters: {'n_estimators': 172, 'max_depth': 12, 'min_samples_split': 13, 'min_samples_leaf': 1, 'max_features': 'sqrt'}. Best is trial 82 with value: 0.8967391304347826.
[I 2023-07-31 17:03:49,245] Trial 83 finished with value: 0.8967391304347826 and parameters: {'n_estimators': 205, 'max_depth': 11, 'min_samples_split': 12, 'min_samples_leaf': 1, 'max_features': 'sqrt'}. Best is trial 82 with value: 0.8967391304347826.
[I 2023-07-31 17:03:49,379] Trial 84 finished with value: 0.875 and parameters: {'n_estimators': 100, 'max_depth': 16, 'min_samples_split': 12, 'min_samples_leaf': 1, 'max_features': 'sqrt'}. Best is trial 82 with value: 0.8967391304347826.
[I 2023-07-31 17:03:49,618] Trial 85 finished with value: 0.842391304347826 and parameters: {'n_estimators': 216, 'max_depth': 13, 'min_samples_split': 14, 'min_samples_leaf': 15, 'max_features': 'sqrt'}. Best is trial 82 with value: 0.8967391304347826.
[I 2023-07-31 17:03:49,794] Trial 86 finished with value: 0.8641304347826086 and parameters: {'n_estimators': 139, 'max_depth': 9, 'min_samples_split': 13, 'min_samples_leaf': 3, 'max_features': 'auto'}. Best is trial 82 with value: 0.8967391304347826.
[I 2023-07-31 17:03:50,015] Trial 87 finished with value: 0.8478260869565217 and parameters: {'n_estimators': 201, 'max_depth': 10, 'min_samples_split': 12, 'min_samples_leaf': 20, 'max_features': 'sqrt'}. Best is trial 82 with value: 0.8967391304347826.
[I 2023-07-31 17:03:50,317] Trial 88 finished with value: 0.8695652173913043 and parameters: {'n_estimators': 270, 'max_depth': 12, 'min_samples_split': 15, 'min_samples_leaf': 4, 'max_features': 'sqrt'}. Best is trial 82 with value: 0.8967391304347826.
[I 2023-07-31 17:03:50,499] Trial 89 finished with value: 0.875 and parameters: {'n_estimators': 151, 'max_depth': 8, 'min_samples_split': 10, 'min_samples_leaf': 1, 'max_features': 'sqrt'}. Best is trial 82 with value: 0.8967391304347826.
[I 2023-07-31 17:03:50,934] Trial 90 finished with value: 0.8478260869565217 and parameters: {'n_estimators': 431, 'max_depth': 15, 'min_samples_split': 11, 'min_samples_leaf': 19, 'max_features': 'sqrt'}. Best is trial 82 with value: 0.8967391304347826.
[I 2023-07-31 17:03:51,089] Trial 91 finished with value: 0.8858695652173914 and parameters: {'n_estimators': 116, 'max_depth': 11, 'min_samples_split': 14, 'min_samples_leaf': 1, 'max_features': 'sqrt'}. Best is trial 82 with value: 0.8967391304347826.
[I 2023-07-31 17:03:51,308] Trial 92 finished with value: 0.875 and parameters: {'n_estimators': 189, 'max_depth': 11, 'min_samples_split': 13, 'min_samples_leaf': 2, 'max_features': 'sqrt'}. Best is trial 82 with value: 0.8967391304347826.
[I 2023-07-31 17:03:51,577] Trial 93 finished with value: 0.875 and parameters: {'n_estimators': 231, 'max_depth': 14, 'min_samples_split': 11, 'min_samples_leaf': 1, 'max_features': 'sqrt'}. Best is trial 82 with value: 0.8967391304347826.
[I 2023-07-31 17:03:51,775] Trial 94 finished with value: 0.8695652173913043 and parameters: {'n_estimators': 165, 'max_depth': 9, 'min_samples_split': 8, 'min_samples_leaf': 2, 'max_features': 'sqrt'}. Best is trial 82 with value: 0.8967391304347826.
[I 2023-07-31 17:03:52,578] Trial 95 finished with value: 0.8858695652173914 and parameters: {'n_estimators': 761, 'max_depth': 7, 'min_samples_split': 12, 'min_samples_leaf': 1, 'max_features': 'sqrt'}. Best is trial 82 with value: 0.8967391304347826.
[I 2023-07-31 17:03:52,741] Trial 96 finished with value: 0.8695652173913043 and parameters: {'n_estimators': 127, 'max_depth': 10, 'min_samples_split': 10, 'min_samples_leaf': 2, 'max_features': 'sqrt'}. Best is trial 82 with value: 0.8967391304347826.
[I 2023-07-31 17:03:52,904] Trial 97 finished with value: 0.8478260869565217 and parameters: {'n_estimators': 144, 'max_depth': 12, 'min_samples_split': 9, 'min_samples_leaf': 16, 'max_features': 'log2'}. Best is trial 82 with value: 0.8967391304347826.
[I 2023-07-31 17:03:53,120] Trial 98 finished with value: 0.8641304347826086 and parameters: {'n_estimators': 191, 'max_depth': 6, 'min_samples_split': 7, 'min_samples_leaf': 3, 'max_features': 'sqrt'}. Best is trial 82 with value: 0.8967391304347826.
[I 2023-07-31 17:03:53,268] Trial 99 finished with value: 0.8858695652173914 and parameters: {'n_estimators': 116, 'max_depth': 8, 'min_samples_split': 2, 'min_samples_leaf': 1, 'max_features': 'sqrt'}. Best is trial 82 with value: 0.8967391304347826.

Best Hyperparameters: {'n_estimators': 172, 'max_depth': 12, 'min_samples_split': 13, 'min_samples_leaf': 1, 'max_features': 'sqrt'}
Best Score (Accuracy): 0.8967391304347826

LGBM Classifier Hyperparams

X = newDf.drop('HeartDisease', axis=1)
y = newDf['HeartDisease']

def objective(trial):
    # Define the hyperparameter search space
    params = {
        "n_estimators": trial.suggest_int("n_estimators", 100, 1000),
        "learning_rate": trial.suggest_loguniform("learning_rate", 0.01, 0.3),
        "max_depth": trial.suggest_int("max_depth", 1, 10),
        "reg_lambda": trial.suggest_loguniform("reg_lambda", 1e-9, 100),
    }
    
    # Split the data into train and validation sets
    X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=42)

    # Create and train the LGBMClassifier with the suggested hyperparameters
    model = LGBMClassifier(**params)
    model.fit(X_train, y_train, eval_set=[(X_val, y_val)], early_stopping_rounds=50, verbose=False)

    # Calculate the evaluation metric to optimize (e.g., accuracy)
    accuracy = model.score(X_val, y_val)

    return accuracy

# Create a study object and optimize the objective function
study = optuna.create_study(direction="maximize")  # We want to maximize the accuracy
study.optimize(objective, n_trials=100)

# Get the best hyperparameters and their corresponding score
best_params = study.best_params
best_score = study.best_value

print("Best Hyperparameters:", best_params)
print("Best Score (Accuracy):", best_score)

[I 2023-07-31 17:03:53,283] A new study created in memory with name: no-name-e57bd1c2-648c-4fd6-9d1d-2c73e223be49
[I 2023-07-31 17:03:53,397] Trial 0 finished with value: 0.8532608695652174 and parameters: {'n_estimators': 267, 'learning_rate': 0.013018153556057203, 'max_depth': 5, 'reg_lambda': 1.2227625048456115e-06}. Best is trial 0 with value: 0.8532608695652174.
[I 2023-07-31 17:03:53,568] Trial 1 finished with value: 0.8478260869565217 and parameters: {'n_estimators': 827, 'learning_rate': 0.013681151154548918, 'max_depth': 8, 'reg_lambda': 2.71101632345767e-07}. Best is trial 0 with value: 0.8532608695652174.
[I 2023-07-31 17:03:53,661] Trial 2 finished with value: 0.8695652173913043 and parameters: {'n_estimators': 376, 'learning_rate': 0.01534352336333233, 'max_depth': 5, 'reg_lambda': 6.1679725988130635e-06}. Best is trial 2 with value: 0.8695652173913043.
[I 2023-07-31 17:03:53,768] Trial 3 finished with value: 0.8695652173913043 and parameters: {'n_estimators': 714, 'learning_rate': 0.01082081852215703, 'max_depth': 2, 'reg_lambda': 2.8166926867938007}. Best is trial 2 with value: 0.8695652173913043.
[I 2023-07-31 17:03:53,788] Trial 4 finished with value: 0.8804347826086957 and parameters: {'n_estimators': 256, 'learning_rate': 0.29431297856200556, 'max_depth': 2, 'reg_lambda': 0.01756369905707436}. Best is trial 4 with value: 0.8804347826086957.
[I 2023-07-31 17:03:53,837] Trial 5 finished with value: 0.8532608695652174 and parameters: {'n_estimators': 766, 'learning_rate': 0.07349556158196278, 'max_depth': 8, 'reg_lambda': 8.773195073832356e-09}. Best is trial 4 with value: 0.8804347826086957.
[I 2023-07-31 17:03:53,889] Trial 6 finished with value: 0.8695652173913043 and parameters: {'n_estimators': 345, 'learning_rate': 0.09470782437266827, 'max_depth': 10, 'reg_lambda': 3.7816861336781533e-05}. Best is trial 4 with value: 0.8804347826086957.
[I 2023-07-31 17:03:53,918] Trial 7 finished with value: 0.8695652173913043 and parameters: {'n_estimators': 212, 'learning_rate': 0.11712390364765193, 'max_depth': 3, 'reg_lambda': 0.022451523787321695}. Best is trial 4 with value: 0.8804347826086957.
[I 2023-07-31 17:03:53,939] Trial 8 finished with value: 0.8641304347826086 and parameters: {'n_estimators': 998, 'learning_rate': 0.2417010135422515, 'max_depth': 3, 'reg_lambda': 0.499071014922435}. Best is trial 4 with value: 0.8804347826086957.
[I 2023-07-31 17:03:53,969] Trial 9 finished with value: 0.875 and parameters: {'n_estimators': 845, 'learning_rate': 0.11771991614406213, 'max_depth': 2, 'reg_lambda': 0.19301740271160522}. Best is trial 4 with value: 0.8804347826086957.
[I 2023-07-31 17:03:54,012] Trial 10 finished with value: 0.8641304347826086 and parameters: {'n_estimators': 111, 'learning_rate': 0.29997884423916543, 'max_depth': 1, 'reg_lambda': 0.0008510150777079021}. Best is trial 4 with value: 0.8804347826086957.
[I 2023-07-31 17:03:54,058] Trial 11 finished with value: 0.8695652173913043 and parameters: {'n_estimators': 517, 'learning_rate': 0.2185399023423723, 'max_depth': 1, 'reg_lambda': 11.30311184771701}. Best is trial 4 with value: 0.8804347826086957.
[I 2023-07-31 17:03:54,104] Trial 12 finished with value: 0.8695652173913043 and parameters: {'n_estimators': 554, 'learning_rate': 0.15495548961868058, 'max_depth': 4, 'reg_lambda': 0.010375843074132848}. Best is trial 4 with value: 0.8804347826086957.
[I 2023-07-31 17:03:54,163] Trial 13 finished with value: 0.8695652173913043 and parameters: {'n_estimators': 551, 'learning_rate': 0.1649388555304257, 'max_depth': 7, 'reg_lambda': 0.12708035735027287}. Best is trial 4 with value: 0.8804347826086957.
[I 2023-07-31 17:03:54,300] Trial 14 finished with value: 0.8804347826086957 and parameters: {'n_estimators': 977, 'learning_rate': 0.04951384313688183, 'max_depth': 3, 'reg_lambda': 45.419771035345455}. Best is trial 4 with value: 0.8804347826086957.
[I 2023-07-31 17:03:54,417] Trial 15 finished with value: 0.875 and parameters: {'n_estimators': 942, 'learning_rate': 0.03902661537988936, 'max_depth': 3, 'reg_lambda': 16.808597307160888}. Best is trial 4 with value: 0.8804347826086957.
[I 2023-07-31 17:03:54,583] Trial 16 finished with value: 0.875 and parameters: {'n_estimators': 454, 'learning_rate': 0.046985560427766075, 'max_depth': 6, 'reg_lambda': 38.25507124409767}. Best is trial 4 with value: 0.8804347826086957.
[I 2023-07-31 17:03:54,643] Trial 17 finished with value: 0.8695652173913043 and parameters: {'n_estimators': 639, 'learning_rate': 0.06301976388432276, 'max_depth': 4, 'reg_lambda': 0.0015664923091507127}. Best is trial 4 with value: 0.8804347826086957.
[I 2023-07-31 17:03:54,691] Trial 18 finished with value: 0.8641304347826086 and parameters: {'n_estimators': 117, 'learning_rate': 0.03200675512022005, 'max_depth': 2, 'reg_lambda': 63.4319298988081}. Best is trial 4 with value: 0.8804347826086957.
[I 2023-07-31 17:03:54,784] Trial 19 finished with value: 0.8804347826086957 and parameters: {'n_estimators': 402, 'learning_rate': 0.028608478663222206, 'max_depth': 4, 'reg_lambda': 1.406445888909495}. Best is trial 4 with value: 0.8804347826086957.
[I 2023-07-31 17:03:54,841] Trial 20 finished with value: 0.8641304347826086 and parameters: {'n_estimators': 662, 'learning_rate': 0.07507596283611886, 'max_depth': 1, 'reg_lambda': 0.03259968075403107}. Best is trial 4 with value: 0.8804347826086957.
[I 2023-07-31 17:03:54,922] Trial 21 finished with value: 0.8695652173913043 and parameters: {'n_estimators': 381, 'learning_rate': 0.02946623422200741, 'max_depth': 4, 'reg_lambda': 0.7409071709520275}. Best is trial 4 with value: 0.8804347826086957.
[I 2023-07-31 17:03:54,991] Trial 22 finished with value: 0.8695652173913043 and parameters: {'n_estimators': 249, 'learning_rate': 0.049567964277917015, 'max_depth': 3, 'reg_lambda': 1.5522513313185486}. Best is trial 4 with value: 0.8804347826086957.
[I 2023-07-31 17:03:55,143] Trial 23 finished with value: 0.875 and parameters: {'n_estimators': 466, 'learning_rate': 0.023851109367003704, 'max_depth': 6, 'reg_lambda': 94.36857901353164}. Best is trial 4 with value: 0.8804347826086957.
[I 2023-07-31 17:03:55,258] Trial 24 finished with value: 0.8858695652173914 and parameters: {'n_estimators': 305, 'learning_rate': 0.02118547949447521, 'max_depth': 4, 'reg_lambda': 5.997345032230852}. Best is trial 24 with value: 0.8858695652173914.
[I 2023-07-31 17:03:55,330] Trial 25 finished with value: 0.875 and parameters: {'n_estimators': 302, 'learning_rate': 0.022363531436930862, 'max_depth': 2, 'reg_lambda': 6.320702914593107}. Best is trial 24 with value: 0.8858695652173914.
[I 2023-07-31 17:03:55,416] Trial 26 finished with value: 0.8967391304347826 and parameters: {'n_estimators': 185, 'learning_rate': 0.05630880792841062, 'max_depth': 5, 'reg_lambda': 4.687239071813298}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:55,526] Trial 27 finished with value: 0.8586956521739131 and parameters: {'n_estimators': 194, 'learning_rate': 0.01771876655807937, 'max_depth': 5, 'reg_lambda': 0.13834464019407935}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:55,620] Trial 28 finished with value: 0.8695652173913043 and parameters: {'n_estimators': 163, 'learning_rate': 0.03802543362610596, 'max_depth': 7, 'reg_lambda': 5.228744400706692}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:55,755] Trial 29 finished with value: 0.8532608695652174 and parameters: {'n_estimators': 289, 'learning_rate': 0.020451465912424128, 'max_depth': 5, 'reg_lambda': 0.0046023210144541665}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:55,820] Trial 30 finished with value: 0.8641304347826086 and parameters: {'n_estimators': 309, 'learning_rate': 0.06463130155536476, 'max_depth': 5, 'reg_lambda': 0.043402568849406555}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:55,897] Trial 31 finished with value: 0.8858695652173914 and parameters: {'n_estimators': 234, 'learning_rate': 0.04721914230188892, 'max_depth': 3, 'reg_lambda': 13.426694526923475}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:55,985] Trial 32 finished with value: 0.8804347826086957 and parameters: {'n_estimators': 216, 'learning_rate': 0.012446266707254038, 'max_depth': 4, 'reg_lambda': 6.459686711295942}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:56,054] Trial 33 finished with value: 0.8641304347826086 and parameters: {'n_estimators': 154, 'learning_rate': 0.08642024623867389, 'max_depth': 6, 'reg_lambda': 0.3819401477030292}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:56,130] Trial 34 finished with value: 0.8804347826086957 and parameters: {'n_estimators': 287, 'learning_rate': 0.018783509229924344, 'max_depth': 2, 'reg_lambda': 1.7433624815840818}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:56,239] Trial 35 finished with value: 0.8858695652173914 and parameters: {'n_estimators': 250, 'learning_rate': 0.056134245186147756, 'max_depth': 4, 'reg_lambda': 12.259779838968784}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:56,373] Trial 36 finished with value: 0.8804347826086957 and parameters: {'n_estimators': 337, 'learning_rate': 0.059729178458144815, 'max_depth': 5, 'reg_lambda': 17.91486500714344}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:56,460] Trial 37 finished with value: 0.875 and parameters: {'n_estimators': 234, 'learning_rate': 0.03642537268444074, 'max_depth': 4, 'reg_lambda': 2.758504167811863}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:56,536] Trial 38 finished with value: 0.875 and parameters: {'n_estimators': 159, 'learning_rate': 0.014299229911155231, 'max_depth': 10, 'reg_lambda': 18.710061435272255}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:56,684] Trial 39 finished with value: 0.8858695652173914 and parameters: {'n_estimators': 422, 'learning_rate': 0.04461633919793977, 'max_depth': 6, 'reg_lambda': 88.99875941146679}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:56,739] Trial 40 finished with value: 0.8695652173913043 and parameters: {'n_estimators': 366, 'learning_rate': 0.05460573539614581, 'max_depth': 3, 'reg_lambda': 0.4804810918535277}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:56,904] Trial 41 finished with value: 0.8858695652173914 and parameters: {'n_estimators': 435, 'learning_rate': 0.0444431429412412, 'max_depth': 7, 'reg_lambda': 98.97219622015362}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:57,014] Trial 42 finished with value: 0.875 and parameters: {'n_estimators': 252, 'learning_rate': 0.04167972134223788, 'max_depth': 6, 'reg_lambda': 4.257721957415404}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:57,151] Trial 43 finished with value: 0.8804347826086957 and parameters: {'n_estimators': 337, 'learning_rate': 0.054693150269471025, 'max_depth': 8, 'reg_lambda': 26.870594314245317}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:57,248] Trial 44 finished with value: 0.8804347826086957 and parameters: {'n_estimators': 187, 'learning_rate': 0.0693737503282495, 'max_depth': 5, 'reg_lambda': 12.472139004779155}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:57,390] Trial 45 finished with value: 0.8858695652173914 and parameters: {'n_estimators': 406, 'learning_rate': 0.02663579554404477, 'max_depth': 4, 'reg_lambda': 99.98721021225505}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:57,492] Trial 46 finished with value: 0.8858695652173914 and parameters: {'n_estimators': 499, 'learning_rate': 0.03236049395766931, 'max_depth': 6, 'reg_lambda': 2.1983032423141164}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:57,562] Trial 47 finished with value: 0.8858695652173914 and parameters: {'n_estimators': 107, 'learning_rate': 0.03547195448597178, 'max_depth': 7, 'reg_lambda': 8.25617132567962}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:57,664] Trial 48 finished with value: 0.8532608695652174 and parameters: {'n_estimators': 259, 'learning_rate': 0.04335564164137364, 'max_depth': 9, 'reg_lambda': 0.00010538320135956488}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:57,756] Trial 49 finished with value: 0.8804347826086957 and parameters: {'n_estimators': 338, 'learning_rate': 0.07776898870950737, 'max_depth': 3, 'reg_lambda': 25.42514500307172}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:57,859] Trial 50 finished with value: 0.8586956521739131 and parameters: {'n_estimators': 217, 'learning_rate': 0.010019179535152453, 'max_depth': 5, 'reg_lambda': 0.7122107028866116}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:58,026] Trial 51 finished with value: 0.8804347826086957 and parameters: {'n_estimators': 421, 'learning_rate': 0.04316716310959048, 'max_depth': 8, 'reg_lambda': 47.083009902694954}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:58,194] Trial 52 finished with value: 0.8858695652173914 and parameters: {'n_estimators': 448, 'learning_rate': 0.04899598006906323, 'max_depth': 7, 'reg_lambda': 92.99526115720964}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:58,316] Trial 53 finished with value: 0.8858695652173914 and parameters: {'n_estimators': 600, 'learning_rate': 0.054514004851653204, 'max_depth': 7, 'reg_lambda': 9.908756660257872}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:58,450] Trial 54 finished with value: 0.8858695652173914 and parameters: {'n_estimators': 510, 'learning_rate': 0.06026527571238746, 'max_depth': 6, 'reg_lambda': 29.974571054972397}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:58,543] Trial 55 finished with value: 0.8695652173913043 and parameters: {'n_estimators': 366, 'learning_rate': 0.042941489534262633, 'max_depth': 3, 'reg_lambda': 3.7440104116739428}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:58,701] Trial 56 finished with value: 0.8858695652173914 and parameters: {'n_estimators': 431, 'learning_rate': 0.034099477363758304, 'max_depth': 4, 'reg_lambda': 43.06242653891314}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:58,808] Trial 57 finished with value: 0.8858695652173914 and parameters: {'n_estimators': 311, 'learning_rate': 0.038343085997941694, 'max_depth': 9, 'reg_lambda': 1.1903053008574}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:58,913] Trial 58 finished with value: 0.8858695652173914 and parameters: {'n_estimators': 141, 'learning_rate': 0.04943585729909226, 'max_depth': 5, 'reg_lambda': 11.406395588682951}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:59,013] Trial 59 finished with value: 0.8695652173913043 and parameters: {'n_estimators': 194, 'learning_rate': 0.026211200236319416, 'max_depth': 4, 'reg_lambda': 0.31889532057802544}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:59,106] Trial 60 finished with value: 0.8695652173913043 and parameters: {'n_estimators': 472, 'learning_rate': 0.031114117201581148, 'max_depth': 3, 'reg_lambda': 3.6287980585380497}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:59,254] Trial 61 finished with value: 0.8858695652173914 and parameters: {'n_estimators': 393, 'learning_rate': 0.02657058266970445, 'max_depth': 4, 'reg_lambda': 91.43285855980217}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:59,412] Trial 62 finished with value: 0.8858695652173914 and parameters: {'n_estimators': 402, 'learning_rate': 0.046360903250090534, 'max_depth': 5, 'reg_lambda': 84.98809590210043}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:59,563] Trial 63 finished with value: 0.8858695652173914 and parameters: {'n_estimators': 575, 'learning_rate': 0.028714482134820123, 'max_depth': 4, 'reg_lambda': 21.57506782109373}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:59,679] Trial 64 finished with value: 0.8804347826086957 and parameters: {'n_estimators': 275, 'learning_rate': 0.06708830677710821, 'max_depth': 6, 'reg_lambda': 43.64218083530626}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:59,762] Trial 65 finished with value: 0.8586956521739131 and parameters: {'n_estimators': 484, 'learning_rate': 0.058689488728076986, 'max_depth': 2, 'reg_lambda': 8.445482054588288}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:59,868] Trial 66 finished with value: 0.875 and parameters: {'n_estimators': 316, 'learning_rate': 0.03921367449231354, 'max_depth': 4, 'reg_lambda': 1.2120953826891423}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:03:59,989] Trial 67 finished with value: 0.875 and parameters: {'n_estimators': 358, 'learning_rate': 0.03445481224698674, 'max_depth': 3, 'reg_lambda': 43.273154397531435}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:04:00,203] Trial 68 finished with value: 0.8804347826086957 and parameters: {'n_estimators': 531, 'learning_rate': 0.023833538694218108, 'max_depth': 4, 'reg_lambda': 14.465127544751061}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:04:00,347] Trial 69 finished with value: 0.8913043478260869 and parameters: {'n_estimators': 224, 'learning_rate': 0.016997240804218383, 'max_depth': 5, 'reg_lambda': 4.258883198427055}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:04:00,497] Trial 70 finished with value: 0.8532608695652174 and parameters: {'n_estimators': 242, 'learning_rate': 0.015110963349900353, 'max_depth': 5, 'reg_lambda': 0.16648154321977215}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:04:00,623] Trial 71 finished with value: 0.8804347826086957 and parameters: {'n_estimators': 181, 'learning_rate': 0.018046077793194237, 'max_depth': 5, 'reg_lambda': 5.756968436588689}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:04:00,805] Trial 72 finished with value: 0.875 and parameters: {'n_estimators': 757, 'learning_rate': 0.016616447015527853, 'max_depth': 6, 'reg_lambda': 2.7673565803676645}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:04:00,900] Trial 73 finished with value: 0.8858695652173914 and parameters: {'n_estimators': 225, 'learning_rate': 0.020574655634567923, 'max_depth': 7, 'reg_lambda': 96.67905186048618}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:04:01,001] Trial 74 finished with value: 0.8858695652173914 and parameters: {'n_estimators': 261, 'learning_rate': 0.053506012422817065, 'max_depth': 4, 'reg_lambda': 29.034232737209344}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:04:01,073] Trial 75 finished with value: 0.8641304347826086 and parameters: {'n_estimators': 133, 'learning_rate': 0.012292024468774638, 'max_depth': 5, 'reg_lambda': 19.253400348349373}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:04:01,270] Trial 76 finished with value: 0.875 and parameters: {'n_estimators': 425, 'learning_rate': 0.016157292035308708, 'max_depth': 6, 'reg_lambda': 5.805969096497799}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:04:01,390] Trial 77 finished with value: 0.8804347826086957 and parameters: {'n_estimators': 283, 'learning_rate': 0.020948396535966682, 'max_depth': 4, 'reg_lambda': 1.9595251931959021}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:04:01,490] Trial 78 finished with value: 0.8695652173913043 and parameters: {'n_estimators': 204, 'learning_rate': 0.045230389534434076, 'max_depth': 7, 'reg_lambda': 54.83497410043113}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:04:01,567] Trial 79 finished with value: 0.875 and parameters: {'n_estimators': 173, 'learning_rate': 0.038823340754060756, 'max_depth': 3, 'reg_lambda': 13.171959999647283}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:04:01,645] Trial 80 finished with value: 0.8641304347826086 and parameters: {'n_estimators': 894, 'learning_rate': 0.050123450720525695, 'max_depth': 5, 'reg_lambda': 0.07791409833708904}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:04:01,756] Trial 81 finished with value: 0.8858695652173914 and parameters: {'n_estimators': 488, 'learning_rate': 0.03325945168049673, 'max_depth': 6, 'reg_lambda': 2.3700386639332187}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:04:01,864] Trial 82 finished with value: 0.8641304347826086 and parameters: {'n_estimators': 450, 'learning_rate': 0.03130492800125262, 'max_depth': 6, 'reg_lambda': 0.8784954441627809}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:04:02,053] Trial 83 finished with value: 0.8967391304347826 and parameters: {'n_estimators': 383, 'learning_rate': 0.041006133372181446, 'max_depth': 8, 'reg_lambda': 8.047033187336476}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:04:02,316] Trial 84 finished with value: 0.8913043478260869 and parameters: {'n_estimators': 313, 'learning_rate': 0.04099131939481015, 'max_depth': 8, 'reg_lambda': 7.562002199660309}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:04:02,419] Trial 85 finished with value: 0.875 and parameters: {'n_estimators': 326, 'learning_rate': 0.06244651342894715, 'max_depth': 8, 'reg_lambda': 4.627680292669212}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:04:02,559] Trial 86 finished with value: 0.8913043478260869 and parameters: {'n_estimators': 289, 'learning_rate': 0.0402494944795181, 'max_depth': 8, 'reg_lambda': 8.121831880205434}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:04:02,656] Trial 87 finished with value: 0.842391304347826 and parameters: {'n_estimators': 295, 'learning_rate': 0.03622532231099927, 'max_depth': 8, 'reg_lambda': 0.6275773218164925}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:04:02,771] Trial 88 finished with value: 0.8804347826086957 and parameters: {'n_estimators': 234, 'learning_rate': 0.040942646082876226, 'max_depth': 9, 'reg_lambda': 7.737290309343424}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:04:02,859] Trial 89 finished with value: 0.8695652173913043 and parameters: {'n_estimators': 273, 'learning_rate': 0.052296266782023274, 'max_depth': 8, 'reg_lambda': 1.6563015053998353}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:04:02,961] Trial 90 finished with value: 0.8641304347826086 and parameters: {'n_estimators': 350, 'learning_rate': 0.04625475559716627, 'max_depth': 9, 'reg_lambda': 0.282071451077157}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:04:03,159] Trial 91 finished with value: 0.8858695652173914 and parameters: {'n_estimators': 385, 'learning_rate': 0.04081943485275355, 'max_depth': 8, 'reg_lambda': 27.345357193111564}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:04:03,318] Trial 92 finished with value: 0.875 and parameters: {'n_estimators': 298, 'learning_rate': 0.058576421448379185, 'max_depth': 7, 'reg_lambda': 12.9228974569045}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:04:03,423] Trial 93 finished with value: 0.875 and parameters: {'n_estimators': 373, 'learning_rate': 0.04441357882498328, 'max_depth': 8, 'reg_lambda': 3.4744496827215685}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:04:03,554] Trial 94 finished with value: 0.875 and parameters: {'n_estimators': 212, 'learning_rate': 0.04730650091231224, 'max_depth': 7, 'reg_lambda': 7.163018590273796}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:04:03,677] Trial 95 finished with value: 0.8913043478260869 and parameters: {'n_estimators': 324, 'learning_rate': 0.05129354561820014, 'max_depth': 8, 'reg_lambda': 49.07134233184788}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:04:03,798] Trial 96 finished with value: 0.875 and parameters: {'n_estimators': 324, 'learning_rate': 0.05679328238583312, 'max_depth': 9, 'reg_lambda': 18.67726258828204}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:04:03,901] Trial 97 finished with value: 0.8804347826086957 and parameters: {'n_estimators': 257, 'learning_rate': 0.05162505197093655, 'max_depth': 8, 'reg_lambda': 57.07669424016409}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:04:03,979] Trial 98 finished with value: 0.8532608695652174 and parameters: {'n_estimators': 240, 'learning_rate': 0.07035142336465472, 'max_depth': 10, 'reg_lambda': 0.8961087309625484}. Best is trial 26 with value: 0.8967391304347826.
[I 2023-07-31 17:04:04,118] Trial 99 finished with value: 0.8858695652173914 and parameters: {'n_estimators': 341, 'learning_rate': 0.036691213385776994, 'max_depth': 9, 'reg_lambda': 33.97978479144624}. Best is trial 26 with value: 0.8967391304347826.

Best Hyperparameters: {'n_estimators': 185, 'learning_rate': 0.05630880792841062, 'max_depth': 5, 'reg_lambda': 4.687239071813298}
Best Score (Accuracy): 0.8967391304347826

from sklearn.ensemble import RandomForestClassifier
from catboost import CatBoostClassifier
from lightgbm import LGBMClassifier
from sklearn.model_selection import train_test_split

# Split the data into train and validation sets
X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=42)

# Define the individual classifiers
#{'n_estimators': 373, 'max_depth': 10, 'min_samples_split': 10, 'min_samples_leaf': 1, 'max_features': 'log2'}
rf_classifier = RandomForestClassifier(random_state=42, n_estimators=373, max_depth=10, min_samples_split=10, min_samples_leaf=1, max_features='log2')
#{'iterations': 542, 'learning_rate': 0.21231296216531056, 'depth': 6, 'l2_leaf_reg': 13.675507885439055}
catboost_classifier = CatBoostClassifier(random_state=42, iterations=542, learning_rate=0.21231296216531056, depth=6, l2_leaf_reg=13.675507885439055)
#{'n_estimators': 637, 'learning_rate': 0.11625626039818515, 'max_depth': 7, 'reg_lambda': 4.777261357298674}
lgbm_classifier = LGBMClassifier(random_state=42, n_estimators=637, learning_rate=0.11625626039818515, max_depth=7, reg_lambda=4.777261357298674)

# Define the bagging classifier, combining the three individual classifiers
bagging_clf = BaggingClassifier(
    base_estimator=VotingClassifier(
        estimators=[
            ('rf', rf_classifier),
            ('catboost', catboost_classifier),
            ('lgbm', lgbm_classifier)
        ],
        voting='hard'  # Voting type 'hard' for majority voting
    ),
    n_estimators=10,  # Number of base estimators
    random_state=42
)

# Train the bagging classifier
bagging_clf.fit(X_train, y_train)

# Evaluate the bagging classifier on the validation set
accuracy = bagging_clf.score(X_val, y_val)

print("Bagging Classifier Accuracy:", accuracy)

0:	learn: 0.5498791	total: 2.09ms	remaining: 1.13s
1:	learn: 0.4450642	total: 3.81ms	remaining: 1.03s
2:	learn: 0.3819276	total: 5.64ms	remaining: 1.01s
3:	learn: 0.3396926	total: 7.55ms	remaining: 1.01s
4:	learn: 0.3103555	total: 9.6ms	remaining: 1.03s
5:	learn: 0.2923378	total: 11.4ms	remaining: 1.02s
6:	learn: 0.2711111	total: 13.3ms	remaining: 1.02s
7:	learn: 0.2524003	total: 14.9ms	remaining: 995ms
8:	learn: 0.2401941	total: 16.7ms	remaining: 986ms
9:	learn: 0.2260957	total: 18.5ms	remaining: 982ms
10:	learn: 0.2149582	total: 20.3ms	remaining: 978ms
11:	learn: 0.2088944	total: 21.8ms	remaining: 961ms
12:	learn: 0.2017182	total: 23.4ms	remaining: 951ms
13:	learn: 0.1931806	total: 25.5ms	remaining: 962ms
14:	learn: 0.1863846	total: 27.9ms	remaining: 981ms
15:	learn: 0.1820839	total: 30.1ms	remaining: 991ms
16:	learn: 0.1749746	total: 57.3ms	remaining: 1.77s
17:	learn: 0.1678627	total: 59.7ms	remaining: 1.74s
18:	learn: 0.1643671	total: 61.5ms	remaining: 1.69s
19:	learn: 0.1624191	total: 63.2ms	remaining: 1.65s
20:	learn: 0.1603981	total: 65.4ms	remaining: 1.62s
21:	learn: 0.1552558	total: 67.6ms	remaining: 1.6s
22:	learn: 0.1510474	total: 70.2ms	remaining: 1.58s
23:	learn: 0.1481917	total: 72.2ms	remaining: 1.56s
24:	learn: 0.1445299	total: 74ms	remaining: 1.53s
25:	learn: 0.1429082	total: 75.7ms	remaining: 1.5s
26:	learn: 0.1378898	total: 77.6ms	remaining: 1.48s
27:	learn: 0.1341018	total: 79.2ms	remaining: 1.45s
28:	learn: 0.1326609	total: 80.9ms	remaining: 1.43s
29:	learn: 0.1303405	total: 82.4ms	remaining: 1.41s
30:	learn: 0.1261269	total: 84.1ms	remaining: 1.39s
31:	learn: 0.1225254	total: 85.9ms	remaining: 1.37s
32:	learn: 0.1198314	total: 87.9ms	remaining: 1.35s
33:	learn: 0.1173934	total: 89.7ms	remaining: 1.34s
34:	learn: 0.1144599	total: 91.4ms	remaining: 1.32s
35:	learn: 0.1119401	total: 93.1ms	remaining: 1.31s
36:	learn: 0.1086234	total: 94.8ms	remaining: 1.29s
37:	learn: 0.1060941	total: 96.6ms	remaining: 1.28s
38:	learn: 0.1043862	total: 98.3ms	remaining: 1.27s
39:	learn: 0.1010934	total: 100ms	remaining: 1.26s
40:	learn: 0.0987693	total: 102ms	remaining: 1.24s
41:	learn: 0.0962964	total: 103ms	remaining: 1.23s
42:	learn: 0.0934958	total: 105ms	remaining: 1.22s
43:	learn: 0.0900870	total: 106ms	remaining: 1.2s
44:	learn: 0.0866827	total: 108ms	remaining: 1.2s
45:	learn: 0.0849804	total: 110ms	remaining: 1.19s
46:	learn: 0.0833356	total: 112ms	remaining: 1.18s
47:	learn: 0.0823007	total: 114ms	remaining: 1.17s
48:	learn: 0.0813094	total: 115ms	remaining: 1.16s
49:	learn: 0.0807264	total: 117ms	remaining: 1.15s
50:	learn: 0.0787701	total: 119ms	remaining: 1.14s
51:	learn: 0.0774283	total: 120ms	remaining: 1.13s
52:	learn: 0.0761870	total: 122ms	remaining: 1.12s
53:	learn: 0.0742248	total: 123ms	remaining: 1.11s
54:	learn: 0.0719370	total: 125ms	remaining: 1.11s
55:	learn: 0.0698432	total: 127ms	remaining: 1.1s
56:	learn: 0.0685025	total: 129ms	remaining: 1.1s
57:	learn: 0.0666136	total: 131ms	remaining: 1.09s
58:	learn: 0.0655044	total: 133ms	remaining: 1.09s
59:	learn: 0.0645107	total: 135ms	remaining: 1.09s
60:	learn: 0.0627137	total: 137ms	remaining: 1.08s
61:	learn: 0.0613639	total: 140ms	remaining: 1.08s
62:	learn: 0.0605148	total: 141ms	remaining: 1.07s
63:	learn: 0.0592217	total: 143ms	remaining: 1.07s
64:	learn: 0.0578289	total: 145ms	remaining: 1.07s
65:	learn: 0.0570790	total: 147ms	remaining: 1.06s
66:	learn: 0.0559313	total: 149ms	remaining: 1.06s
67:	learn: 0.0546932	total: 151ms	remaining: 1.05s
68:	learn: 0.0532912	total: 153ms	remaining: 1.05s
69:	learn: 0.0527073	total: 155ms	remaining: 1.04s
70:	learn: 0.0515468	total: 157ms	remaining: 1.04s
71:	learn: 0.0506335	total: 159ms	remaining: 1.04s
72:	learn: 0.0494199	total: 160ms	remaining: 1.03s
73:	learn: 0.0487928	total: 162ms	remaining: 1.03s
74:	learn: 0.0476036	total: 164ms	remaining: 1.02s
75:	learn: 0.0471364	total: 167ms	remaining: 1.02s
76:	learn: 0.0464449	total: 168ms	remaining: 1.02s
77:	learn: 0.0458273	total: 171ms	remaining: 1.01s
78:	learn: 0.0450817	total: 173ms	remaining: 1.01s
79:	learn: 0.0446137	total: 175ms	remaining: 1.01s
80:	learn: 0.0443245	total: 176ms	remaining: 1s
81:	learn: 0.0440666	total: 178ms	remaining: 1s
82:	learn: 0.0434965	total: 180ms	remaining: 996ms
83:	learn: 0.0428808	total: 182ms	remaining: 992ms
84:	learn: 0.0422616	total: 183ms	remaining: 986ms
85:	learn: 0.0414400	total: 185ms	remaining: 981ms
86:	learn: 0.0411645	total: 187ms	remaining: 978ms
87:	learn: 0.0402862	total: 189ms	remaining: 973ms
88:	learn: 0.0394754	total: 190ms	remaining: 969ms
89:	learn: 0.0393364	total: 192ms	remaining: 965ms
90:	learn: 0.0386260	total: 194ms	remaining: 963ms
91:	learn: 0.0380961	total: 196ms	remaining: 959ms
92:	learn: 0.0378923	total: 198ms	remaining: 957ms
93:	learn: 0.0373957	total: 200ms	remaining: 954ms
94:	learn: 0.0366906	total: 202ms	remaining: 951ms
95:	learn: 0.0365816	total: 204ms	remaining: 947ms
96:	learn: 0.0362962	total: 205ms	remaining: 943ms
97:	learn: 0.0355803	total: 207ms	remaining: 939ms
98:	learn: 0.0351842	total: 209ms	remaining: 934ms
99:	learn: 0.0345176	total: 210ms	remaining: 930ms
100:	learn: 0.0341008	total: 212ms	remaining: 926ms
101:	learn: 0.0335164	total: 214ms	remaining: 922ms
102:	learn: 0.0328290	total: 216ms	remaining: 919ms
103:	learn: 0.0325438	total: 218ms	remaining: 919ms
104:	learn: 0.0319293	total: 229ms	remaining: 953ms
105:	learn: 0.0313911	total: 231ms	remaining: 952ms
106:	learn: 0.0311152	total: 233ms	remaining: 949ms
107:	learn: 0.0308501	total: 235ms	remaining: 945ms
108:	learn: 0.0303566	total: 237ms	remaining: 941ms
109:	learn: 0.0301229	total: 239ms	remaining: 938ms
110:	learn: 0.0297626	total: 241ms	remaining: 935ms
111:	learn: 0.0295985	total: 242ms	remaining: 931ms
112:	learn: 0.0290547	total: 245ms	remaining: 928ms
113:	learn: 0.0287131	total: 246ms	remaining: 925ms
114:	learn: 0.0286015	total: 248ms	remaining: 922ms
115:	learn: 0.0284608	total: 250ms	remaining: 919ms
116:	learn: 0.0284005	total: 252ms	remaining: 916ms
117:	learn: 0.0281758	total: 254ms	remaining: 914ms
118:	learn: 0.0278883	total: 257ms	remaining: 913ms
119:	learn: 0.0276940	total: 259ms	remaining: 910ms
120:	learn: 0.0273049	total: 261ms	remaining: 908ms
121:	learn: 0.0270972	total: 263ms	remaining: 906ms
122:	learn: 0.0267486	total: 265ms	remaining: 904ms
123:	learn: 0.0265922	total: 268ms	remaining: 903ms
124:	learn: 0.0263965	total: 270ms	remaining: 900ms
125:	learn: 0.0260145	total: 272ms	remaining: 899ms
126:	learn: 0.0258605	total: 274ms	remaining: 896ms
127:	learn: 0.0256067	total: 276ms	remaining: 893ms
128:	learn: 0.0252934	total: 278ms	remaining: 891ms
129:	learn: 0.0251527	total: 281ms	remaining: 889ms
130:	learn: 0.0248246	total: 283ms	remaining: 888ms
131:	learn: 0.0244836	total: 285ms	remaining: 887ms
132:	learn: 0.0241206	total: 288ms	remaining: 884ms
133:	learn: 0.0239426	total: 289ms	remaining: 881ms
134:	learn: 0.0236870	total: 292ms	remaining: 879ms
135:	learn: 0.0236264	total: 294ms	remaining: 876ms
136:	learn: 0.0234407	total: 296ms	remaining: 874ms
137:	learn: 0.0232511	total: 298ms	remaining: 871ms
138:	learn: 0.0230417	total: 300ms	remaining: 869ms
139:	learn: 0.0229300	total: 301ms	remaining: 865ms
140:	learn: 0.0227859	total: 303ms	remaining: 861ms
141:	learn: 0.0226741	total: 304ms	remaining: 858ms
142:	learn: 0.0224731	total: 306ms	remaining: 854ms
143:	learn: 0.0224524	total: 308ms	remaining: 850ms
144:	learn: 0.0222658	total: 310ms	remaining: 847ms
145:	learn: 0.0221346	total: 315ms	remaining: 854ms
146:	learn: 0.0218077	total: 316ms	remaining: 850ms
147:	learn: 0.0215820	total: 318ms	remaining: 847ms
148:	learn: 0.0214291	total: 320ms	remaining: 843ms
149:	learn: 0.0213463	total: 321ms	remaining: 840ms
150:	learn: 0.0212338	total: 323ms	remaining: 836ms
151:	learn: 0.0209308	total: 324ms	remaining: 832ms
152:	learn: 0.0206840	total: 326ms	remaining: 829ms
153:	learn: 0.0204179	total: 328ms	remaining: 825ms
154:	learn: 0.0202491	total: 329ms	remaining: 822ms
155:	learn: 0.0201426	total: 331ms	remaining: 819ms
156:	learn: 0.0200235	total: 333ms	remaining: 815ms
157:	learn: 0.0199386	total: 334ms	remaining: 812ms
158:	learn: 0.0197754	total: 336ms	remaining: 809ms
159:	learn: 0.0196935	total: 337ms	remaining: 805ms
160:	learn: 0.0196112	total: 339ms	remaining: 803ms
161:	learn: 0.0195366	total: 341ms	remaining: 800ms
162:	learn: 0.0193427	total: 343ms	remaining: 798ms
163:	learn: 0.0191473	total: 346ms	remaining: 796ms
164:	learn: 0.0190205	total: 347ms	remaining: 794ms
165:	learn: 0.0188175	total: 350ms	remaining: 792ms
166:	learn: 0.0186737	total: 352ms	remaining: 790ms
167:	learn: 0.0185066	total: 354ms	remaining: 788ms
168:	learn: 0.0183603	total: 356ms	remaining: 786ms
169:	learn: 0.0182284	total: 359ms	remaining: 784ms
170:	learn: 0.0179818	total: 361ms	remaining: 782ms
171:	learn: 0.0177968	total: 363ms	remaining: 781ms
172:	learn: 0.0177946	total: 365ms	remaining: 779ms
173:	learn: 0.0176645	total: 367ms	remaining: 777ms
174:	learn: 0.0174957	total: 369ms	remaining: 775ms
175:	learn: 0.0174238	total: 372ms	remaining: 773ms
176:	learn: 0.0173823	total: 375ms	remaining: 773ms
177:	learn: 0.0172389	total: 377ms	remaining: 772ms
178:	learn: 0.0170961	total: 380ms	remaining: 770ms
179:	learn: 0.0169965	total: 382ms	remaining: 769ms
180:	learn: 0.0168179	total: 384ms	remaining: 766ms
181:	learn: 0.0167105	total: 387ms	remaining: 765ms
182:	learn: 0.0166110	total: 389ms	remaining: 763ms
183:	learn: 0.0164435	total: 391ms	remaining: 760ms
184:	learn: 0.0163170	total: 393ms	remaining: 759ms
185:	learn: 0.0163087	total: 395ms	remaining: 757ms
186:	learn: 0.0162328	total: 397ms	remaining: 754ms
187:	learn: 0.0161532	total: 399ms	remaining: 752ms
188:	learn: 0.0160569	total: 402ms	remaining: 750ms
189:	learn: 0.0159766	total: 405ms	remaining: 750ms
190:	learn: 0.0158342	total: 407ms	remaining: 747ms
191:	learn: 0.0158040	total: 409ms	remaining: 745ms
192:	learn: 0.0156747	total: 411ms	remaining: 744ms
193:	learn: 0.0155224	total: 413ms	remaining: 741ms
194:	learn: 0.0154978	total: 415ms	remaining: 739ms
195:	learn: 0.0154131	total: 417ms	remaining: 736ms
196:	learn: 0.0153022	total: 419ms	remaining: 735ms
197:	learn: 0.0151789	total: 421ms	remaining: 732ms
198:	learn: 0.0150624	total: 423ms	remaining: 729ms
199:	learn: 0.0148970	total: 425ms	remaining: 726ms
200:	learn: 0.0148668	total: 426ms	remaining: 723ms
201:	learn: 0.0147692	total: 428ms	remaining: 721ms
202:	learn: 0.0145937	total: 430ms	remaining: 718ms
203:	learn: 0.0144599	total: 432ms	remaining: 716ms
204:	learn: 0.0143734	total: 434ms	remaining: 714ms
205:	learn: 0.0142767	total: 436ms	remaining: 711ms
206:	learn: 0.0141985	total: 437ms	remaining: 708ms
207:	learn: 0.0141228	total: 444ms	remaining: 713ms
208:	learn: 0.0140408	total: 446ms	remaining: 711ms
209:	learn: 0.0140172	total: 448ms	remaining: 708ms
210:	learn: 0.0139360	total: 449ms	remaining: 705ms
211:	learn: 0.0138244	total: 451ms	remaining: 702ms
212:	learn: 0.0136866	total: 452ms	remaining: 699ms
213:	learn: 0.0136205	total: 454ms	remaining: 696ms
214:	learn: 0.0135128	total: 456ms	remaining: 693ms
215:	learn: 0.0134930	total: 457ms	remaining: 690ms
216:	learn: 0.0133793	total: 459ms	remaining: 688ms
217:	learn: 0.0132949	total: 461ms	remaining: 685ms
218:	learn: 0.0132946	total: 463ms	remaining: 683ms
219:	learn: 0.0131948	total: 464ms	remaining: 680ms
220:	learn: 0.0131518	total: 466ms	remaining: 677ms
221:	learn: 0.0131115	total: 468ms	remaining: 674ms
222:	learn: 0.0130153	total: 470ms	remaining: 672ms
223:	learn: 0.0129505	total: 471ms	remaining: 669ms
224:	learn: 0.0128907	total: 473ms	remaining: 666ms
225:	learn: 0.0127611	total: 474ms	remaining: 663ms
226:	learn: 0.0127220	total: 476ms	remaining: 660ms
227:	learn: 0.0126196	total: 477ms	remaining: 658ms
228:	learn: 0.0125454	total: 479ms	remaining: 655ms
229:	learn: 0.0124998	total: 481ms	remaining: 652ms
230:	learn: 0.0124460	total: 483ms	remaining: 650ms
231:	learn: 0.0123692	total: 484ms	remaining: 647ms
232:	learn: 0.0122858	total: 486ms	remaining: 644ms
233:	learn: 0.0121886	total: 487ms	remaining: 642ms
234:	learn: 0.0121034	total: 489ms	remaining: 639ms
235:	learn: 0.0120713	total: 491ms	remaining: 636ms
236:	learn: 0.0119915	total: 492ms	remaining: 633ms
237:	learn: 0.0119099	total: 494ms	remaining: 631ms
238:	learn: 0.0118097	total: 496ms	remaining: 628ms
239:	learn: 0.0117089	total: 497ms	remaining: 626ms
240:	learn: 0.0116303	total: 499ms	remaining: 623ms
241:	learn: 0.0115897	total: 501ms	remaining: 621ms
242:	learn: 0.0115370	total: 503ms	remaining: 619ms
243:	learn: 0.0114906	total: 505ms	remaining: 617ms
244:	learn: 0.0114591	total: 507ms	remaining: 614ms
245:	learn: 0.0114109	total: 509ms	remaining: 612ms
246:	learn: 0.0113282	total: 510ms	remaining: 609ms
247:	learn: 0.0112579	total: 512ms	remaining: 607ms
248:	learn: 0.0112051	total: 514ms	remaining: 604ms
249:	learn: 0.0111481	total: 515ms	remaining: 602ms
250:	learn: 0.0110605	total: 517ms	remaining: 600ms
251:	learn: 0.0109803	total: 519ms	remaining: 597ms
252:	learn: 0.0109086	total: 521ms	remaining: 595ms
253:	learn: 0.0108440	total: 522ms	remaining: 592ms
254:	learn: 0.0108030	total: 524ms	remaining: 589ms
255:	learn: 0.0107586	total: 526ms	remaining: 587ms
256:	learn: 0.0107168	total: 527ms	remaining: 584ms
257:	learn: 0.0106796	total: 529ms	remaining: 582ms
258:	learn: 0.0106510	total: 530ms	remaining: 580ms
259:	learn: 0.0105805	total: 532ms	remaining: 577ms
260:	learn: 0.0105167	total: 534ms	remaining: 575ms
261:	learn: 0.0104865	total: 536ms	remaining: 573ms
262:	learn: 0.0104374	total: 538ms	remaining: 570ms
263:	learn: 0.0103978	total: 540ms	remaining: 568ms
264:	learn: 0.0103648	total: 541ms	remaining: 566ms
265:	learn: 0.0103257	total: 543ms	remaining: 563ms
266:	learn: 0.0102896	total: 545ms	remaining: 561ms
267:	learn: 0.0102497	total: 547ms	remaining: 559ms
268:	learn: 0.0102071	total: 548ms	remaining: 557ms
269:	learn: 0.0101759	total: 550ms	remaining: 554ms
270:	learn: 0.0101334	total: 552ms	remaining: 552ms
271:	learn: 0.0100722	total: 554ms	remaining: 550ms
272:	learn: 0.0100382	total: 555ms	remaining: 547ms
273:	learn: 0.0100049	total: 557ms	remaining: 545ms
274:	learn: 0.0099518	total: 559ms	remaining: 542ms
275:	learn: 0.0099121	total: 560ms	remaining: 540ms
276:	learn: 0.0098895	total: 562ms	remaining: 537ms
277:	learn: 0.0098689	total: 564ms	remaining: 535ms
278:	learn: 0.0098164	total: 565ms	remaining: 533ms
279:	learn: 0.0097614	total: 568ms	remaining: 531ms
280:	learn: 0.0097141	total: 570ms	remaining: 529ms
281:	learn: 0.0096746	total: 571ms	remaining: 527ms
282:	learn: 0.0096263	total: 573ms	remaining: 525ms
283:	learn: 0.0095905	total: 575ms	remaining: 522ms
284:	learn: 0.0095696	total: 577ms	remaining: 520ms
285:	learn: 0.0095399	total: 578ms	remaining: 518ms
286:	learn: 0.0094845	total: 580ms	remaining: 515ms
287:	learn: 0.0094477	total: 582ms	remaining: 513ms
288:	learn: 0.0094477	total: 583ms	remaining: 511ms
289:	learn: 0.0094270	total: 585ms	remaining: 508ms
290:	learn: 0.0093635	total: 586ms	remaining: 506ms
291:	learn: 0.0093164	total: 588ms	remaining: 503ms
292:	learn: 0.0092653	total: 590ms	remaining: 501ms
293:	learn: 0.0092346	total: 592ms	remaining: 499ms
294:	learn: 0.0092177	total: 594ms	remaining: 498ms
295:	learn: 0.0092176	total: 596ms	remaining: 495ms
296:	learn: 0.0091860	total: 598ms	remaining: 493ms
297:	learn: 0.0091565	total: 600ms	remaining: 491ms
298:	learn: 0.0091310	total: 602ms	remaining: 489ms
299:	learn: 0.0090675	total: 604ms	remaining: 487ms
300:	learn: 0.0090392	total: 606ms	remaining: 485ms
301:	learn: 0.0089977	total: 608ms	remaining: 483ms
302:	learn: 0.0089711	total: 610ms	remaining: 481ms
303:	learn: 0.0089252	total: 612ms	remaining: 479ms
304:	learn: 0.0088844	total: 613ms	remaining: 477ms
305:	learn: 0.0088508	total: 615ms	remaining: 474ms
306:	learn: 0.0088260	total: 617ms	remaining: 472ms
307:	learn: 0.0087746	total: 619ms	remaining: 470ms
308:	learn: 0.0087366	total: 620ms	remaining: 468ms
309:	learn: 0.0087130	total: 622ms	remaining: 466ms
310:	learn: 0.0086941	total: 624ms	remaining: 463ms
311:	learn: 0.0086587	total: 626ms	remaining: 461ms
312:	learn: 0.0086333	total: 628ms	remaining: 460ms
313:	learn: 0.0086224	total: 630ms	remaining: 457ms
314:	learn: 0.0085631	total: 632ms	remaining: 456ms
315:	learn: 0.0085411	total: 634ms	remaining: 454ms
316:	learn: 0.0085233	total: 636ms	remaining: 452ms
317:	learn: 0.0085053	total: 638ms	remaining: 450ms
318:	learn: 0.0084924	total: 641ms	remaining: 448ms
319:	learn: 0.0084453	total: 643ms	remaining: 446ms
320:	learn: 0.0084176	total: 644ms	remaining: 443ms
321:	learn: 0.0083733	total: 646ms	remaining: 441ms
322:	learn: 0.0083364	total: 647ms	remaining: 439ms
323:	learn: 0.0083056	total: 649ms	remaining: 437ms
324:	learn: 0.0082706	total: 651ms	remaining: 435ms
325:	learn: 0.0082546	total: 652ms	remaining: 432ms
326:	learn: 0.0082060	total: 654ms	remaining: 430ms
327:	learn: 0.0081851	total: 656ms	remaining: 428ms
328:	learn: 0.0081616	total: 658ms	remaining: 426ms
329:	learn: 0.0081207	total: 660ms	remaining: 424ms
330:	learn: 0.0080996	total: 662ms	remaining: 422ms
331:	learn: 0.0080796	total: 663ms	remaining: 419ms
332:	learn: 0.0080222	total: 665ms	remaining: 417ms
333:	learn: 0.0080036	total: 667ms	remaining: 415ms
334:	learn: 0.0079877	total: 668ms	remaining: 413ms
335:	learn: 0.0079529	total: 670ms	remaining: 411ms
336:	learn: 0.0079387	total: 672ms	remaining: 409ms
337:	learn: 0.0079241	total: 673ms	remaining: 406ms
338:	learn: 0.0078904	total: 675ms	remaining: 404ms
339:	learn: 0.0078594	total: 677ms	remaining: 402ms
340:	learn: 0.0078594	total: 678ms	remaining: 400ms
341:	learn: 0.0078235	total: 679ms	remaining: 397ms
342:	learn: 0.0077847	total: 681ms	remaining: 395ms
343:	learn: 0.0077578	total: 683ms	remaining: 393ms
344:	learn: 0.0077359	total: 685ms	remaining: 391ms
345:	learn: 0.0077018	total: 687ms	remaining: 389ms
346:	learn: 0.0076854	total: 688ms	remaining: 387ms
347:	learn: 0.0076572	total: 690ms	remaining: 385ms
348:	learn: 0.0076572	total: 691ms	remaining: 382ms
349:	learn: 0.0076214	total: 693ms	remaining: 380ms
350:	learn: 0.0076004	total: 695ms	remaining: 378ms
351:	learn: 0.0076004	total: 696ms	remaining: 376ms
352:	learn: 0.0075738	total: 698ms	remaining: 374ms
353:	learn: 0.0075380	total: 699ms	remaining: 371ms
354:	learn: 0.0075058	total: 701ms	remaining: 369ms
355:	learn: 0.0074766	total: 703ms	remaining: 367ms
356:	learn: 0.0074451	total: 705ms	remaining: 365ms
357:	learn: 0.0074200	total: 706ms	remaining: 363ms
358:	learn: 0.0073893	total: 708ms	remaining: 361ms
359:	learn: 0.0073616	total: 710ms	remaining: 359ms
360:	learn: 0.0073407	total: 711ms	remaining: 356ms
361:	learn: 0.0073029	total: 713ms	remaining: 354ms
362:	learn: 0.0072710	total: 714ms	remaining: 352ms
363:	learn: 0.0072467	total: 716ms	remaining: 350ms
364:	learn: 0.0072267	total: 718ms	remaining: 348ms
365:	learn: 0.0071964	total: 720ms	remaining: 346ms
366:	learn: 0.0071689	total: 722ms	remaining: 344ms
367:	learn: 0.0071493	total: 724ms	remaining: 342ms
368:	learn: 0.0071309	total: 726ms	remaining: 340ms
369:	learn: 0.0071094	total: 728ms	remaining: 338ms
370:	learn: 0.0070990	total: 730ms	remaining: 336ms
371:	learn: 0.0070990	total: 732ms	remaining: 334ms
372:	learn: 0.0070714	total: 734ms	remaining: 332ms
373:	learn: 0.0070380	total: 735ms	remaining: 330ms
374:	learn: 0.0070380	total: 737ms	remaining: 328ms
375:	learn: 0.0070377	total: 738ms	remaining: 326ms
376:	learn: 0.0070376	total: 740ms	remaining: 324ms
377:	learn: 0.0070376	total: 741ms	remaining: 322ms
378:	learn: 0.0070377	total: 743ms	remaining: 319ms
379:	learn: 0.0070375	total: 744ms	remaining: 317ms
380:	learn: 0.0070375	total: 746ms	remaining: 315ms
381:	learn: 0.0070375	total: 747ms	remaining: 313ms
382:	learn: 0.0070375	total: 749ms	remaining: 311ms
383:	learn: 0.0070375	total: 764ms	remaining: 314ms
384:	learn: 0.0070375	total: 766ms	remaining: 312ms
385:	learn: 0.0070374	total: 767ms	remaining: 310ms
386:	learn: 0.0070374	total: 768ms	remaining: 308ms
387:	learn: 0.0070374	total: 770ms	remaining: 306ms
388:	learn: 0.0070374	total: 771ms	remaining: 303ms
389:	learn: 0.0070374	total: 772ms	remaining: 301ms
390:	learn: 0.0070374	total: 774ms	remaining: 299ms
391:	learn: 0.0070374	total: 775ms	remaining: 297ms
392:	learn: 0.0070374	total: 777ms	remaining: 295ms
393:	learn: 0.0070372	total: 779ms	remaining: 293ms
394:	learn: 0.0070120	total: 782ms	remaining: 291ms
395:	learn: 0.0070120	total: 784ms	remaining: 289ms
396:	learn: 0.0070119	total: 785ms	remaining: 287ms
397:	learn: 0.0070119	total: 787ms	remaining: 285ms
398:	learn: 0.0070118	total: 789ms	remaining: 283ms
399:	learn: 0.0070117	total: 790ms	remaining: 281ms
400:	learn: 0.0070117	total: 792ms	remaining: 278ms
401:	learn: 0.0070117	total: 794ms	remaining: 276ms
402:	learn: 0.0070118	total: 795ms	remaining: 274ms
403:	learn: 0.0070118	total: 796ms	remaining: 272ms
404:	learn: 0.0070118	total: 798ms	remaining: 270ms
405:	learn: 0.0070118	total: 799ms	remaining: 268ms
406:	learn: 0.0070118	total: 801ms	remaining: 266ms
407:	learn: 0.0070116	total: 802ms	remaining: 263ms
408:	learn: 0.0070117	total: 804ms	remaining: 261ms
409:	learn: 0.0070117	total: 805ms	remaining: 259ms
410:	learn: 0.0070117	total: 807ms	remaining: 257ms
411:	learn: 0.0070117	total: 809ms	remaining: 255ms
412:	learn: 0.0070117	total: 810ms	remaining: 253ms
413:	learn: 0.0070116	total: 812ms	remaining: 251ms
414:	learn: 0.0070116	total: 814ms	remaining: 249ms
415:	learn: 0.0070116	total: 816ms	remaining: 247ms
416:	learn: 0.0070115	total: 818ms	remaining: 245ms
417:	learn: 0.0070116	total: 819ms	remaining: 243ms
418:	learn: 0.0070115	total: 821ms	remaining: 241ms
419:	learn: 0.0070115	total: 823ms	remaining: 239ms
420:	learn: 0.0070116	total: 824ms	remaining: 237ms
421:	learn: 0.0070116	total: 826ms	remaining: 235ms
422:	learn: 0.0070116	total: 827ms	remaining: 233ms
423:	learn: 0.0070114	total: 829ms	remaining: 231ms
424:	learn: 0.0070115	total: 830ms	remaining: 228ms
425:	learn: 0.0070113	total: 832ms	remaining: 226ms
426:	learn: 0.0070113	total: 833ms	remaining: 224ms
427:	learn: 0.0070113	total: 834ms	remaining: 222ms
428:	learn: 0.0070113	total: 836ms	remaining: 220ms
429:	learn: 0.0070113	total: 838ms	remaining: 218ms
430:	learn: 0.0070113	total: 839ms	remaining: 216ms
431:	learn: 0.0070112	total: 840ms	remaining: 214ms
432:	learn: 0.0070112	total: 842ms	remaining: 212ms
433:	learn: 0.0070112	total: 843ms	remaining: 210ms
434:	learn: 0.0070111	total: 845ms	remaining: 208ms
435:	learn: 0.0070111	total: 846ms	remaining: 206ms
436:	learn: 0.0070110	total: 848ms	remaining: 204ms
437:	learn: 0.0070111	total: 849ms	remaining: 202ms
438:	learn: 0.0070111	total: 850ms	remaining: 200ms
439:	learn: 0.0070109	total: 852ms	remaining: 197ms
440:	learn: 0.0070108	total: 853ms	remaining: 195ms
441:	learn: 0.0070107	total: 855ms	remaining: 193ms
442:	learn: 0.0070107	total: 856ms	remaining: 191ms
443:	learn: 0.0070107	total: 858ms	remaining: 189ms
444:	learn: 0.0070107	total: 859ms	remaining: 187ms
445:	learn: 0.0070108	total: 860ms	remaining: 185ms
446:	learn: 0.0070108	total: 862ms	remaining: 183ms
447:	learn: 0.0070108	total: 863ms	remaining: 181ms
448:	learn: 0.0070108	total: 865ms	remaining: 179ms
449:	learn: 0.0070107	total: 866ms	remaining: 177ms
450:	learn: 0.0070106	total: 868ms	remaining: 175ms
451:	learn: 0.0070106	total: 869ms	remaining: 173ms
452:	learn: 0.0070106	total: 870ms	remaining: 171ms
453:	learn: 0.0070105	total: 872ms	remaining: 169ms
454:	learn: 0.0070105	total: 873ms	remaining: 167ms
455:	learn: 0.0070104	total: 875ms	remaining: 165ms
456:	learn: 0.0070104	total: 876ms	remaining: 163ms
457:	learn: 0.0070104	total: 878ms	remaining: 161ms
458:	learn: 0.0070103	total: 879ms	remaining: 159ms
459:	learn: 0.0070103	total: 881ms	remaining: 157ms
460:	learn: 0.0070103	total: 883ms	remaining: 155ms
461:	learn: 0.0070102	total: 884ms	remaining: 153ms
462:	learn: 0.0070102	total: 886ms	remaining: 151ms
463:	learn: 0.0070102	total: 887ms	remaining: 149ms
464:	learn: 0.0070102	total: 889ms	remaining: 147ms
465:	learn: 0.0070101	total: 890ms	remaining: 145ms
466:	learn: 0.0070100	total: 892ms	remaining: 143ms
467:	learn: 0.0070100	total: 894ms	remaining: 141ms
468:	learn: 0.0070100	total: 896ms	remaining: 139ms
469:	learn: 0.0070099	total: 909ms	remaining: 139ms
470:	learn: 0.0070098	total: 911ms	remaining: 137ms
471:	learn: 0.0070099	total: 912ms	remaining: 135ms
472:	learn: 0.0070099	total: 914ms	remaining: 133ms
473:	learn: 0.0070097	total: 915ms	remaining: 131ms
474:	learn: 0.0070096	total: 917ms	remaining: 129ms
475:	learn: 0.0070095	total: 919ms	remaining: 127ms
476:	learn: 0.0070095	total: 920ms	remaining: 125ms
477:	learn: 0.0070095	total: 922ms	remaining: 123ms
478:	learn: 0.0070093	total: 924ms	remaining: 121ms
479:	learn: 0.0070093	total: 925ms	remaining: 119ms
480:	learn: 0.0070093	total: 926ms	remaining: 117ms
481:	learn: 0.0070092	total: 928ms	remaining: 115ms
482:	learn: 0.0070091	total: 929ms	remaining: 114ms
483:	learn: 0.0070091	total: 931ms	remaining: 112ms
484:	learn: 0.0070090	total: 932ms	remaining: 110ms
485:	learn: 0.0070089	total: 934ms	remaining: 108ms
486:	learn: 0.0070088	total: 935ms	remaining: 106ms
487:	learn: 0.0070088	total: 937ms	remaining: 104ms
488:	learn: 0.0070088	total: 938ms	remaining: 102ms
489:	learn: 0.0070087	total: 939ms	remaining: 99.7ms
490:	learn: 0.0070087	total: 941ms	remaining: 97.7ms
491:	learn: 0.0070087	total: 942ms	remaining: 95.8ms
492:	learn: 0.0070086	total: 944ms	remaining: 93.8ms
493:	learn: 0.0070086	total: 946ms	remaining: 91.9ms
494:	learn: 0.0070085	total: 947ms	remaining: 89.9ms
495:	learn: 0.0070084	total: 949ms	remaining: 88ms
496:	learn: 0.0070084	total: 951ms	remaining: 86.1ms
497:	learn: 0.0070084	total: 952ms	remaining: 84.1ms
498:	learn: 0.0070082	total: 954ms	remaining: 82.2ms
499:	learn: 0.0070081	total: 956ms	remaining: 80.3ms
500:	learn: 0.0070081	total: 957ms	remaining: 78.3ms
501:	learn: 0.0070080	total: 958ms	remaining: 76.4ms
502:	learn: 0.0070080	total: 960ms	remaining: 74.4ms
503:	learn: 0.0070080	total: 961ms	remaining: 72.5ms
504:	learn: 0.0070080	total: 962ms	remaining: 70.5ms
505:	learn: 0.0070080	total: 964ms	remaining: 68.6ms
506:	learn: 0.0070080	total: 965ms	remaining: 66.6ms
507:	learn: 0.0070080	total: 966ms	remaining: 64.7ms
508:	learn: 0.0070080	total: 968ms	remaining: 62.7ms
509:	learn: 0.0070080	total: 969ms	remaining: 60.8ms
510:	learn: 0.0070080	total: 971ms	remaining: 58.9ms
511:	learn: 0.0070080	total: 972ms	remaining: 57ms
512:	learn: 0.0070080	total: 974ms	remaining: 55ms
513:	learn: 0.0070080	total: 975ms	remaining: 53.1ms
514:	learn: 0.0070080	total: 976ms	remaining: 51.2ms
515:	learn: 0.0070080	total: 978ms	remaining: 49.3ms
516:	learn: 0.0070080	total: 979ms	remaining: 47.3ms
517:	learn: 0.0070080	total: 980ms	remaining: 45.4ms
518:	learn: 0.0070080	total: 982ms	remaining: 43.5ms
519:	learn: 0.0070080	total: 983ms	remaining: 41.6ms
520:	learn: 0.0070080	total: 984ms	remaining: 39.7ms
521:	learn: 0.0070080	total: 985ms	remaining: 37.8ms
522:	learn: 0.0070080	total: 987ms	remaining: 35.8ms
523:	learn: 0.0070080	total: 988ms	remaining: 33.9ms
524:	learn: 0.0070080	total: 989ms	remaining: 32ms
525:	learn: 0.0070080	total: 991ms	remaining: 30.1ms
526:	learn: 0.0070080	total: 992ms	remaining: 28.2ms
527:	learn: 0.0070080	total: 994ms	remaining: 26.3ms
528:	learn: 0.0070080	total: 995ms	remaining: 24.5ms
529:	learn: 0.0070080	total: 997ms	remaining: 22.6ms
530:	learn: 0.0070080	total: 998ms	remaining: 20.7ms
531:	learn: 0.0070080	total: 999ms	remaining: 18.8ms
532:	learn: 0.0070080	total: 1s	remaining: 16.9ms
533:	learn: 0.0070080	total: 1s	remaining: 15ms
534:	learn: 0.0070080	total: 1s	remaining: 13.1ms
535:	learn: 0.0070080	total: 1.01s	remaining: 11.3ms
536:	learn: 0.0070080	total: 1.01s	remaining: 9.38ms
537:	learn: 0.0070080	total: 1.01s	remaining: 7.5ms
538:	learn: 0.0070080	total: 1.01s	remaining: 5.63ms
539:	learn: 0.0070080	total: 1.01s	remaining: 3.75ms
540:	learn: 0.0070080	total: 1.01s	remaining: 1.87ms
541:	learn: 0.0070080	total: 1.01s	remaining: 0us
0:	learn: 0.5491996	total: 2.14ms	remaining: 1.16s
1:	learn: 0.4584767	total: 4.81ms	remaining: 1.3s
2:	learn: 0.4086197	total: 6.79ms	remaining: 1.22s
3:	learn: 0.3718177	total: 9.03ms	remaining: 1.21s
4:	learn: 0.3444828	total: 11.1ms	remaining: 1.19s
5:	learn: 0.3215668	total: 12.9ms	remaining: 1.16s
6:	learn: 0.3100053	total: 14.7ms	remaining: 1.12s
7:	learn: 0.2943616	total: 16.4ms	remaining: 1.1s
8:	learn: 0.2835224	total: 18.2ms	remaining: 1.07s
9:	learn: 0.2678337	total: 32.3ms	remaining: 1.72s
10:	learn: 0.2547700	total: 34.1ms	remaining: 1.65s
11:	learn: 0.2446138	total: 36ms	remaining: 1.59s
12:	learn: 0.2375026	total: 37.8ms	remaining: 1.54s
13:	learn: 0.2333748	total: 39.5ms	remaining: 1.49s
14:	learn: 0.2249006	total: 41ms	remaining: 1.44s
15:	learn: 0.2209905	total: 42.6ms	remaining: 1.4s
16:	learn: 0.2168927	total: 44.3ms	remaining: 1.37s
17:	learn: 0.2111873	total: 45.8ms	remaining: 1.33s
18:	learn: 0.2080679	total: 47.7ms	remaining: 1.31s
19:	learn: 0.2019865	total: 49.3ms	remaining: 1.28s
20:	learn: 0.1978650	total: 51ms	remaining: 1.27s
21:	learn: 0.1954345	total: 52.8ms	remaining: 1.25s
22:	learn: 0.1905390	total: 55.1ms	remaining: 1.24s
23:	learn: 0.1871165	total: 56.8ms	remaining: 1.23s
24:	learn: 0.1801434	total: 58.5ms	remaining: 1.21s
25:	learn: 0.1759724	total: 60.4ms	remaining: 1.2s
26:	learn: 0.1743086	total: 62.1ms	remaining: 1.19s
27:	learn: 0.1710756	total: 63.9ms	remaining: 1.17s
28:	learn: 0.1679764	total: 65.6ms	remaining: 1.16s
29:	learn: 0.1658229	total: 67.3ms	remaining: 1.15s
30:	learn: 0.1621832	total: 69ms	remaining: 1.14s
31:	learn: 0.1564367	total: 70.7ms	remaining: 1.13s
32:	learn: 0.1544909	total: 72.3ms	remaining: 1.11s
33:	learn: 0.1512800	total: 74.1ms	remaining: 1.11s
34:	learn: 0.1476900	total: 75.8ms	remaining: 1.1s
35:	learn: 0.1452492	total: 77.3ms	remaining: 1.08s
36:	learn: 0.1437919	total: 79.2ms	remaining: 1.08s
37:	learn: 0.1427559	total: 80.7ms	remaining: 1.07s
38:	learn: 0.1382236	total: 82.7ms	remaining: 1.07s
39:	learn: 0.1343272	total: 84.9ms	remaining: 1.06s
40:	learn: 0.1292781	total: 87ms	remaining: 1.06s
41:	learn: 0.1258115	total: 88.8ms	remaining: 1.06s
42:	learn: 0.1216495	total: 90.5ms	remaining: 1.05s
43:	learn: 0.1183274	total: 92.3ms	remaining: 1.04s
44:	learn: 0.1156021	total: 94.3ms	remaining: 1.04s
45:	learn: 0.1144258	total: 96.1ms	remaining: 1.04s
46:	learn: 0.1115544	total: 98ms	remaining: 1.03s
47:	learn: 0.1092263	total: 99.9ms	remaining: 1.03s
48:	learn: 0.1078875	total: 102ms	remaining: 1.03s
49:	learn: 0.1051193	total: 104ms	remaining: 1.02s
50:	learn: 0.1019427	total: 105ms	remaining: 1.01s
51:	learn: 0.0993342	total: 107ms	remaining: 1.01s
52:	learn: 0.0960944	total: 109ms	remaining: 1s
53:	learn: 0.0952106	total: 110ms	remaining: 997ms
54:	learn: 0.0926787	total: 112ms	remaining: 992ms
55:	learn: 0.0905204	total: 114ms	remaining: 988ms
56:	learn: 0.0881470	total: 116ms	remaining: 983ms
57:	learn: 0.0862586	total: 117ms	remaining: 979ms
58:	learn: 0.0854314	total: 119ms	remaining: 974ms
59:	learn: 0.0830519	total: 121ms	remaining: 969ms
60:	learn: 0.0805877	total: 122ms	remaining: 964ms
61:	learn: 0.0797326	total: 124ms	remaining: 958ms
62:	learn: 0.0778578	total: 126ms	remaining: 956ms
63:	learn: 0.0765284	total: 127ms	remaining: 951ms
64:	learn: 0.0754756	total: 129ms	remaining: 949ms
65:	learn: 0.0742010	total: 131ms	remaining: 944ms
66:	learn: 0.0728320	total: 133ms	remaining: 940ms
67:	learn: 0.0722048	total: 134ms	remaining: 936ms
68:	learn: 0.0709477	total: 136ms	remaining: 932ms
69:	learn: 0.0706947	total: 138ms	remaining: 929ms
70:	learn: 0.0702756	total: 139ms	remaining: 924ms
71:	learn: 0.0700332	total: 141ms	remaining: 922ms
72:	learn: 0.0691202	total: 143ms	remaining: 918ms
73:	learn: 0.0690192	total: 145ms	remaining: 915ms
74:	learn: 0.0677176	total: 146ms	remaining: 912ms
75:	learn: 0.0658946	total: 148ms	remaining: 909ms
76:	learn: 0.0658347	total: 150ms	remaining: 904ms
77:	learn: 0.0657638	total: 151ms	remaining: 900ms
78:	learn: 0.0645984	total: 153ms	remaining: 898ms
79:	learn: 0.0627635	total: 155ms	remaining: 895ms
80:	learn: 0.0614829	total: 157ms	remaining: 893ms
81:	learn: 0.0597102	total: 158ms	remaining: 889ms
82:	learn: 0.0585821	total: 161ms	remaining: 888ms
83:	learn: 0.0582856	total: 162ms	remaining: 884ms
84:	learn: 0.0572774	total: 164ms	remaining: 880ms
85:	learn: 0.0567529	total: 165ms	remaining: 876ms
86:	learn: 0.0555929	total: 167ms	remaining: 873ms
87:	learn: 0.0549530	total: 169ms	remaining: 870ms
88:	learn: 0.0536640	total: 170ms	remaining: 866ms
89:	learn: 0.0525491	total: 172ms	remaining: 863ms
90:	learn: 0.0518058	total: 174ms	remaining: 860ms
91:	learn: 0.0514010	total: 175ms	remaining: 858ms
92:	learn: 0.0512733	total: 177ms	remaining: 856ms
93:	learn: 0.0507559	total: 179ms	remaining: 853ms
94:	learn: 0.0504256	total: 181ms	remaining: 851ms
95:	learn: 0.0494344	total: 182ms	remaining: 848ms
96:	learn: 0.0488593	total: 184ms	remaining: 845ms
97:	learn: 0.0488266	total: 186ms	remaining: 843ms
98:	learn: 0.0478486	total: 188ms	remaining: 841ms
99:	learn: 0.0472756	total: 190ms	remaining: 838ms
100:	learn: 0.0465664	total: 191ms	remaining: 836ms
101:	learn: 0.0455032	total: 193ms	remaining: 833ms
102:	learn: 0.0452260	total: 195ms	remaining: 829ms
103:	learn: 0.0448881	total: 196ms	remaining: 827ms
104:	learn: 0.0444524	total: 198ms	remaining: 824ms
105:	learn: 0.0440498	total: 200ms	remaining: 821ms
106:	learn: 0.0437790	total: 201ms	remaining: 818ms
107:	learn: 0.0437025	total: 203ms	remaining: 815ms
108:	learn: 0.0429287	total: 205ms	remaining: 813ms
109:	learn: 0.0428161	total: 206ms	remaining: 810ms
110:	learn: 0.0423693	total: 208ms	remaining: 808ms
111:	learn: 0.0422522	total: 210ms	remaining: 805ms
112:	learn: 0.0420165	total: 211ms	remaining: 802ms
113:	learn: 0.0418796	total: 213ms	remaining: 799ms
114:	learn: 0.0413482	total: 214ms	remaining: 796ms
115:	learn: 0.0407233	total: 216ms	remaining: 793ms
116:	learn: 0.0400764	total: 217ms	remaining: 790ms
117:	learn: 0.0392573	total: 219ms	remaining: 787ms
118:	learn: 0.0385676	total: 221ms	remaining: 785ms
119:	learn: 0.0384104	total: 223ms	remaining: 783ms
120:	learn: 0.0382055	total: 224ms	remaining: 781ms
121:	learn: 0.0375090	total: 226ms	remaining: 778ms
122:	learn: 0.0373248	total: 228ms	remaining: 775ms
123:	learn: 0.0372644	total: 229ms	remaining: 772ms
124:	learn: 0.0372370	total: 231ms	remaining: 770ms
125:	learn: 0.0371094	total: 232ms	remaining: 767ms
126:	learn: 0.0368897	total: 234ms	remaining: 764ms
127:	learn: 0.0368695	total: 235ms	remaining: 762ms
128:	learn: 0.0367788	total: 237ms	remaining: 759ms
129:	learn: 0.0363396	total: 239ms	remaining: 756ms
130:	learn: 0.0360322	total: 240ms	remaining: 753ms
131:	learn: 0.0357142	total: 242ms	remaining: 750ms
132:	learn: 0.0354171	total: 243ms	remaining: 748ms
133:	learn: 0.0352904	total: 245ms	remaining: 746ms
134:	learn: 0.0346809	total: 247ms	remaining: 744ms
135:	learn: 0.0341955	total: 248ms	remaining: 742ms
136:	learn: 0.0341798	total: 250ms	remaining: 740ms
137:	learn: 0.0336424	total: 252ms	remaining: 739ms
138:	learn: 0.0331778	total: 254ms	remaining: 737ms
139:	learn: 0.0327663	total: 256ms	remaining: 735ms
140:	learn: 0.0327392	total: 257ms	remaining: 732ms
141:	learn: 0.0326788	total: 259ms	remaining: 730ms
142:	learn: 0.0326254	total: 261ms	remaining: 728ms
143:	learn: 0.0323233	total: 263ms	remaining: 726ms
144:	learn: 0.0322582	total: 264ms	remaining: 724ms
145:	learn: 0.0317843	total: 266ms	remaining: 721ms
146:	learn: 0.0317243	total: 268ms	remaining: 719ms
147:	learn: 0.0316732	total: 269ms	remaining: 716ms
148:	learn: 0.0313265	total: 271ms	remaining: 714ms
149:	learn: 0.0308640	total: 272ms	remaining: 712ms
150:	learn: 0.0302926	total: 274ms	remaining: 710ms
151:	learn: 0.0301973	total: 276ms	remaining: 707ms
152:	learn: 0.0297139	total: 277ms	remaining: 705ms
153:	learn: 0.0292031	total: 279ms	remaining: 704ms
154:	learn: 0.0288717	total: 281ms	remaining: 703ms
155:	learn: 0.0285531	total: 283ms	remaining: 701ms
156:	learn: 0.0285448	total: 285ms	remaining: 699ms
157:	learn: 0.0285367	total: 287ms	remaining: 697ms
158:	learn: 0.0285288	total: 288ms	remaining: 695ms
159:	learn: 0.0285210	total: 290ms	remaining: 693ms
160:	learn: 0.0281244	total: 292ms	remaining: 691ms
161:	learn: 0.0279933	total: 294ms	remaining: 689ms
162:	learn: 0.0278671	total: 295ms	remaining: 687ms
163:	learn: 0.0277441	total: 297ms	remaining: 684ms
164:	learn: 0.0273890	total: 299ms	remaining: 682ms
165:	learn: 0.0270323	total: 300ms	remaining: 680ms
166:	learn: 0.0267658	total: 302ms	remaining: 678ms
167:	learn: 0.0265362	total: 303ms	remaining: 675ms
168:	learn: 0.0264609	total: 305ms	remaining: 673ms
169:	learn: 0.0261123	total: 307ms	remaining: 671ms
170:	learn: 0.0258429	total: 308ms	remaining: 669ms
171:	learn: 0.0256460	total: 310ms	remaining: 667ms
172:	learn: 0.0253749	total: 312ms	remaining: 665ms
173:	learn: 0.0251048	total: 314ms	remaining: 664ms
174:	learn: 0.0249603	total: 316ms	remaining: 662ms
175:	learn: 0.0246855	total: 323ms	remaining: 672ms
176:	learn: 0.0244819	total: 325ms	remaining: 670ms
177:	learn: 0.0243482	total: 326ms	remaining: 667ms
178:	learn: 0.0240683	total: 328ms	remaining: 665ms
179:	learn: 0.0238391	total: 329ms	remaining: 663ms
180:	learn: 0.0234737	total: 331ms	remaining: 660ms
181:	learn: 0.0232366	total: 332ms	remaining: 658ms
182:	learn: 0.0231651	total: 334ms	remaining: 656ms
183:	learn: 0.0230062	total: 336ms	remaining: 654ms
184:	learn: 0.0228281	total: 338ms	remaining: 652ms
185:	learn: 0.0227033	total: 340ms	remaining: 650ms
186:	learn: 0.0225104	total: 342ms	remaining: 648ms
187:	learn: 0.0224243	total: 343ms	remaining: 646ms
188:	learn: 0.0222860	total: 345ms	remaining: 645ms
189:	learn: 0.0220951	total: 347ms	remaining: 642ms
190:	learn: 0.0219317	total: 348ms	remaining: 640ms
191:	learn: 0.0218020	total: 350ms	remaining: 638ms
192:	learn: 0.0217787	total: 352ms	remaining: 636ms
193:	learn: 0.0217541	total: 353ms	remaining: 634ms
194:	learn: 0.0216767	total: 355ms	remaining: 631ms
195:	learn: 0.0214434	total: 357ms	remaining: 630ms
196:	learn: 0.0212828	total: 358ms	remaining: 627ms
197:	learn: 0.0211429	total: 360ms	remaining: 625ms
198:	learn: 0.0210032	total: 361ms	remaining: 623ms
199:	learn: 0.0208148	total: 363ms	remaining: 621ms
200:	learn: 0.0206214	total: 365ms	remaining: 619ms
201:	learn: 0.0204114	total: 366ms	remaining: 617ms
202:	learn: 0.0201603	total: 368ms	remaining: 615ms
203:	learn: 0.0200097	total: 370ms	remaining: 613ms
204:	learn: 0.0198237	total: 372ms	remaining: 611ms
205:	learn: 0.0196778	total: 374ms	remaining: 610ms
206:	learn: 0.0195442	total: 376ms	remaining: 608ms
207:	learn: 0.0194843	total: 378ms	remaining: 606ms
208:	learn: 0.0192832	total: 379ms	remaining: 604ms
209:	learn: 0.0191905	total: 381ms	remaining: 603ms
210:	learn: 0.0191126	total: 383ms	remaining: 601ms
211:	learn: 0.0189343	total: 385ms	remaining: 600ms
212:	learn: 0.0188549	total: 387ms	remaining: 598ms
213:	learn: 0.0187489	total: 389ms	remaining: 596ms
214:	learn: 0.0185988	total: 390ms	remaining: 594ms
215:	learn: 0.0184942	total: 392ms	remaining: 591ms
216:	learn: 0.0183743	total: 394ms	remaining: 589ms
217:	learn: 0.0183716	total: 395ms	remaining: 587ms
218:	learn: 0.0182012	total: 397ms	remaining: 586ms
219:	learn: 0.0181077	total: 399ms	remaining: 584ms
220:	learn: 0.0180019	total: 401ms	remaining: 582ms
221:	learn: 0.0179676	total: 402ms	remaining: 580ms
222:	learn: 0.0179421	total: 404ms	remaining: 578ms
223:	learn: 0.0177593	total: 406ms	remaining: 576ms
224:	learn: 0.0175644	total: 408ms	remaining: 575ms
225:	learn: 0.0174342	total: 410ms	remaining: 573ms
226:	learn: 0.0173300	total: 412ms	remaining: 571ms
227:	learn: 0.0172312	total: 413ms	remaining: 569ms
228:	learn: 0.0172073	total: 415ms	remaining: 567ms
229:	learn: 0.0171363	total: 417ms	remaining: 565ms
230:	learn: 0.0171211	total: 418ms	remaining: 563ms
231:	learn: 0.0169439	total: 420ms	remaining: 561ms
232:	learn: 0.0168994	total: 422ms	remaining: 559ms
233:	learn: 0.0167528	total: 423ms	remaining: 557ms
234:	learn: 0.0166250	total: 425ms	remaining: 555ms
235:	learn: 0.0166091	total: 427ms	remaining: 553ms
236:	learn: 0.0165886	total: 429ms	remaining: 552ms
237:	learn: 0.0165013	total: 430ms	remaining: 550ms
238:	learn: 0.0163812	total: 432ms	remaining: 548ms
239:	learn: 0.0163507	total: 434ms	remaining: 546ms
240:	learn: 0.0163385	total: 436ms	remaining: 544ms
241:	learn: 0.0162554	total: 437ms	remaining: 542ms
242:	learn: 0.0162069	total: 439ms	remaining: 540ms
243:	learn: 0.0161955	total: 441ms	remaining: 538ms
244:	learn: 0.0161624	total: 442ms	remaining: 536ms
245:	learn: 0.0160191	total: 444ms	remaining: 534ms
246:	learn: 0.0159633	total: 445ms	remaining: 532ms
247:	learn: 0.0159478	total: 447ms	remaining: 530ms
248:	learn: 0.0159158	total: 449ms	remaining: 528ms
249:	learn: 0.0157653	total: 450ms	remaining: 526ms
250:	learn: 0.0156864	total: 452ms	remaining: 524ms
251:	learn: 0.0156019	total: 454ms	remaining: 522ms
252:	learn: 0.0155296	total: 455ms	remaining: 520ms
253:	learn: 0.0154777	total: 457ms	remaining: 518ms
254:	learn: 0.0154676	total: 459ms	remaining: 516ms
255:	learn: 0.0153433	total: 460ms	remaining: 514ms
256:	learn: 0.0152946	total: 462ms	remaining: 513ms
257:	learn: 0.0151937	total: 464ms	remaining: 511ms
258:	learn: 0.0150850	total: 466ms	remaining: 509ms
259:	learn: 0.0150325	total: 468ms	remaining: 508ms
260:	learn: 0.0149446	total: 470ms	remaining: 506ms
261:	learn: 0.0148487	total: 472ms	remaining: 505ms
262:	learn: 0.0148391	total: 474ms	remaining: 503ms
263:	learn: 0.0148125	total: 476ms	remaining: 501ms
264:	learn: 0.0147902	total: 477ms	remaining: 499ms
265:	learn: 0.0146636	total: 479ms	remaining: 497ms
266:	learn: 0.0146548	total: 480ms	remaining: 495ms
267:	learn: 0.0146082	total: 482ms	remaining: 493ms
268:	learn: 0.0144882	total: 483ms	remaining: 491ms
269:	learn: 0.0144773	total: 485ms	remaining: 489ms
270:	learn: 0.0144069	total: 487ms	remaining: 487ms
271:	learn: 0.0143268	total: 488ms	remaining: 485ms
272:	learn: 0.0142463	total: 491ms	remaining: 483ms
273:	learn: 0.0141824	total: 492ms	remaining: 482ms
274:	learn: 0.0141383	total: 494ms	remaining: 480ms
275:	learn: 0.0140757	total: 496ms	remaining: 478ms
276:	learn: 0.0140527	total: 512ms	remaining: 490ms
277:	learn: 0.0139931	total: 514ms	remaining: 488ms
278:	learn: 0.0138964	total: 516ms	remaining: 486ms
279:	learn: 0.0138510	total: 517ms	remaining: 484ms
280:	learn: 0.0138109	total: 519ms	remaining: 482ms
281:	learn: 0.0138023	total: 520ms	remaining: 480ms
282:	learn: 0.0136962	total: 522ms	remaining: 478ms
283:	learn: 0.0135948	total: 524ms	remaining: 476ms
284:	learn: 0.0135094	total: 526ms	remaining: 474ms
285:	learn: 0.0134734	total: 528ms	remaining: 472ms
286:	learn: 0.0134734	total: 529ms	remaining: 470ms
287:	learn: 0.0133871	total: 531ms	remaining: 468ms
288:	learn: 0.0133212	total: 533ms	remaining: 466ms
289:	learn: 0.0132540	total: 535ms	remaining: 465ms
290:	learn: 0.0131670	total: 536ms	remaining: 463ms
291:	learn: 0.0131200	total: 538ms	remaining: 461ms
292:	learn: 0.0130621	total: 539ms	remaining: 458ms
293:	learn: 0.0129428	total: 541ms	remaining: 456ms
294:	learn: 0.0128866	total: 543ms	remaining: 454ms
295:	learn: 0.0128731	total: 544ms	remaining: 452ms
296:	learn: 0.0128388	total: 546ms	remaining: 450ms
297:	learn: 0.0127657	total: 547ms	remaining: 448ms
298:	learn: 0.0126913	total: 549ms	remaining: 446ms
299:	learn: 0.0126485	total: 551ms	remaining: 444ms
300:	learn: 0.0126428	total: 553ms	remaining: 442ms
301:	learn: 0.0125777	total: 555ms	remaining: 441ms
302:	learn: 0.0125193	total: 556ms	remaining: 439ms
303:	learn: 0.0125108	total: 558ms	remaining: 437ms
304:	learn: 0.0124864	total: 560ms	remaining: 435ms
305:	learn: 0.0124127	total: 561ms	remaining: 433ms
306:	learn: 0.0124065	total: 563ms	remaining: 431ms
307:	learn: 0.0123603	total: 565ms	remaining: 429ms
308:	learn: 0.0123183	total: 567ms	remaining: 428ms
309:	learn: 0.0122684	total: 569ms	remaining: 426ms
310:	learn: 0.0122174	total: 570ms	remaining: 424ms
311:	learn: 0.0122174	total: 572ms	remaining: 421ms
312:	learn: 0.0121638	total: 573ms	remaining: 419ms
313:	learn: 0.0121485	total: 575ms	remaining: 417ms
314:	learn: 0.0121485	total: 576ms	remaining: 415ms
315:	learn: 0.0121054	total: 578ms	remaining: 413ms
316:	learn: 0.0120666	total: 579ms	remaining: 411ms
317:	learn: 0.0119945	total: 581ms	remaining: 409ms
318:	learn: 0.0119720	total: 583ms	remaining: 408ms
319:	learn: 0.0119669	total: 585ms	remaining: 406ms
320:	learn: 0.0119138	total: 587ms	remaining: 404ms
321:	learn: 0.0118487	total: 589ms	remaining: 402ms
322:	learn: 0.0118241	total: 590ms	remaining: 400ms
323:	learn: 0.0118098	total: 592ms	remaining: 398ms
324:	learn: 0.0117501	total: 594ms	remaining: 396ms
325:	learn: 0.0117389	total: 595ms	remaining: 394ms
326:	learn: 0.0117227	total: 597ms	remaining: 392ms
327:	learn: 0.0116696	total: 598ms	remaining: 390ms
328:	learn: 0.0115965	total: 600ms	remaining: 388ms
329:	learn: 0.0115430	total: 602ms	remaining: 387ms
330:	learn: 0.0115430	total: 603ms	remaining: 384ms
331:	learn: 0.0115343	total: 605ms	remaining: 382ms
332:	learn: 0.0114876	total: 606ms	remaining: 380ms
333:	learn: 0.0114202	total: 608ms	remaining: 379ms
334:	learn: 0.0114143	total: 610ms	remaining: 377ms
335:	learn: 0.0113804	total: 611ms	remaining: 375ms
336:	learn: 0.0113185	total: 613ms	remaining: 373ms
337:	learn: 0.0112461	total: 614ms	remaining: 371ms
338:	learn: 0.0111954	total: 616ms	remaining: 369ms
339:	learn: 0.0111299	total: 618ms	remaining: 367ms
340:	learn: 0.0110504	total: 619ms	remaining: 365ms
341:	learn: 0.0109499	total: 621ms	remaining: 363ms
342:	learn: 0.0108904	total: 623ms	remaining: 361ms
343:	learn: 0.0108391	total: 625ms	remaining: 359ms
344:	learn: 0.0107958	total: 627ms	remaining: 358ms
345:	learn: 0.0107793	total: 629ms	remaining: 356ms
346:	learn: 0.0107250	total: 631ms	remaining: 355ms
347:	learn: 0.0106545	total: 633ms	remaining: 353ms
348:	learn: 0.0105951	total: 634ms	remaining: 351ms
349:	learn: 0.0105569	total: 636ms	remaining: 349ms
350:	learn: 0.0105003	total: 638ms	remaining: 347ms
351:	learn: 0.0104278	total: 640ms	remaining: 345ms
352:	learn: 0.0103763	total: 642ms	remaining: 344ms
353:	learn: 0.0103114	total: 644ms	remaining: 342ms
354:	learn: 0.0102591	total: 645ms	remaining: 340ms
355:	learn: 0.0102588	total: 647ms	remaining: 338ms
356:	learn: 0.0102134	total: 649ms	remaining: 336ms
357:	learn: 0.0101644	total: 650ms	remaining: 334ms
358:	learn: 0.0101185	total: 652ms	remaining: 332ms
359:	learn: 0.0100771	total: 653ms	remaining: 330ms
360:	learn: 0.0100250	total: 655ms	remaining: 329ms
361:	learn: 0.0100081	total: 657ms	remaining: 327ms
362:	learn: 0.0099862	total: 659ms	remaining: 325ms
363:	learn: 0.0099416	total: 661ms	remaining: 323ms
364:	learn: 0.0099229	total: 663ms	remaining: 321ms
365:	learn: 0.0098997	total: 665ms	remaining: 320ms
366:	learn: 0.0098925	total: 666ms	remaining: 318ms
367:	learn: 0.0098447	total: 668ms	remaining: 316ms
368:	learn: 0.0098332	total: 670ms	remaining: 314ms
369:	learn: 0.0098277	total: 671ms	remaining: 312ms
370:	learn: 0.0097985	total: 673ms	remaining: 310ms
371:	learn: 0.0097515	total: 675ms	remaining: 308ms
372:	learn: 0.0097148	total: 677ms	remaining: 307ms
373:	learn: 0.0096722	total: 678ms	remaining: 305ms
374:	learn: 0.0096219	total: 680ms	remaining: 303ms
375:	learn: 0.0095664	total: 681ms	remaining: 301ms
376:	learn: 0.0095276	total: 683ms	remaining: 299ms
377:	learn: 0.0094742	total: 685ms	remaining: 297ms
378:	learn: 0.0094365	total: 687ms	remaining: 295ms
379:	learn: 0.0093930	total: 689ms	remaining: 294ms
380:	learn: 0.0093557	total: 690ms	remaining: 292ms
381:	learn: 0.0093055	total: 692ms	remaining: 290ms
382:	learn: 0.0092601	total: 694ms	remaining: 288ms
383:	learn: 0.0092208	total: 696ms	remaining: 286ms
384:	learn: 0.0091732	total: 698ms	remaining: 285ms
385:	learn: 0.0091266	total: 700ms	remaining: 283ms
386:	learn: 0.0090881	total: 701ms	remaining: 281ms
387:	learn: 0.0090554	total: 703ms	remaining: 279ms
388:	learn: 0.0090324	total: 705ms	remaining: 277ms
389:	learn: 0.0089939	total: 706ms	remaining: 275ms
390:	learn: 0.0089743	total: 708ms	remaining: 273ms
391:	learn: 0.0089477	total: 710ms	remaining: 272ms
392:	learn: 0.0089193	total: 712ms	remaining: 270ms
393:	learn: 0.0088810	total: 713ms	remaining: 268ms
394:	learn: 0.0088638	total: 715ms	remaining: 266ms
395:	learn: 0.0088431	total: 717ms	remaining: 264ms
396:	learn: 0.0088013	total: 719ms	remaining: 263ms
397:	learn: 0.0088003	total: 720ms	remaining: 261ms
398:	learn: 0.0087608	total: 722ms	remaining: 259ms
399:	learn: 0.0087331	total: 724ms	remaining: 257ms
400:	learn: 0.0086931	total: 726ms	remaining: 255ms
401:	learn: 0.0086565	total: 727ms	remaining: 253ms
402:	learn: 0.0086191	total: 729ms	remaining: 251ms
403:	learn: 0.0085883	total: 731ms	remaining: 250ms
404:	learn: 0.0085583	total: 732ms	remaining: 248ms
405:	learn: 0.0085240	total: 734ms	remaining: 246ms
406:	learn: 0.0084878	total: 736ms	remaining: 244ms
407:	learn: 0.0084352	total: 737ms	remaining: 242ms
408:	learn: 0.0083974	total: 739ms	remaining: 240ms
409:	learn: 0.0083974	total: 741ms	remaining: 238ms
410:	learn: 0.0083834	total: 743ms	remaining: 237ms
411:	learn: 0.0083506	total: 744ms	remaining: 235ms
412:	learn: 0.0083384	total: 746ms	remaining: 233ms
413:	learn: 0.0082884	total: 748ms	remaining: 231ms
414:	learn: 0.0082715	total: 750ms	remaining: 230ms
415:	learn: 0.0082424	total: 752ms	remaining: 228ms
416:	learn: 0.0082294	total: 754ms	remaining: 226ms
417:	learn: 0.0081992	total: 756ms	remaining: 224ms
418:	learn: 0.0081680	total: 758ms	remaining: 223ms
419:	learn: 0.0081447	total: 760ms	remaining: 221ms
420:	learn: 0.0081154	total: 762ms	remaining: 219ms
421:	learn: 0.0081153	total: 764ms	remaining: 217ms
422:	learn: 0.0080862	total: 765ms	remaining: 215ms
423:	learn: 0.0080471	total: 767ms	remaining: 213ms
424:	learn: 0.0080201	total: 768ms	remaining: 212ms
425:	learn: 0.0079925	total: 770ms	remaining: 210ms
426:	learn: 0.0079922	total: 772ms	remaining: 208ms
427:	learn: 0.0079678	total: 773ms	remaining: 206ms
428:	learn: 0.0079353	total: 775ms	remaining: 204ms
429:	learn: 0.0079076	total: 777ms	remaining: 202ms
430:	learn: 0.0079076	total: 779ms	remaining: 201ms
431:	learn: 0.0078852	total: 781ms	remaining: 199ms
432:	learn: 0.0078502	total: 782ms	remaining: 197ms
433:	learn: 0.0078211	total: 784ms	remaining: 195ms
434:	learn: 0.0077916	total: 786ms	remaining: 193ms
435:	learn: 0.0077678	total: 788ms	remaining: 192ms
436:	learn: 0.0077393	total: 790ms	remaining: 190ms
437:	learn: 0.0077259	total: 792ms	remaining: 188ms
438:	learn: 0.0077001	total: 794ms	remaining: 186ms
439:	learn: 0.0076748	total: 795ms	remaining: 184ms
440:	learn: 0.0076747	total: 797ms	remaining: 183ms
441:	learn: 0.0076426	total: 799ms	remaining: 181ms
442:	learn: 0.0076115	total: 800ms	remaining: 179ms
443:	learn: 0.0076001	total: 802ms	remaining: 177ms
444:	learn: 0.0075829	total: 804ms	remaining: 175ms
445:	learn: 0.0075558	total: 806ms	remaining: 174ms
446:	learn: 0.0075103	total: 808ms	remaining: 172ms
447:	learn: 0.0074811	total: 810ms	remaining: 170ms
448:	learn: 0.0074583	total: 812ms	remaining: 168ms
449:	learn: 0.0074487	total: 813ms	remaining: 166ms
450:	learn: 0.0074177	total: 815ms	remaining: 164ms
451:	learn: 0.0074043	total: 831ms	remaining: 166ms
452:	learn: 0.0073979	total: 833ms	remaining: 164ms
453:	learn: 0.0073789	total: 835ms	remaining: 162ms
454:	learn: 0.0073478	total: 837ms	remaining: 160ms
455:	learn: 0.0073412	total: 838ms	remaining: 158ms
456:	learn: 0.0073391	total: 846ms	remaining: 157ms
457:	learn: 0.0073207	total: 848ms	remaining: 156ms
458:	learn: 0.0072969	total: 850ms	remaining: 154ms
459:	learn: 0.0072846	total: 852ms	remaining: 152ms
460:	learn: 0.0072669	total: 854ms	remaining: 150ms
461:	learn: 0.0072393	total: 856ms	remaining: 148ms
462:	learn: 0.0072211	total: 857ms	remaining: 146ms
463:	learn: 0.0071998	total: 859ms	remaining: 144ms
464:	learn: 0.0071737	total: 861ms	remaining: 143ms
465:	learn: 0.0071531	total: 863ms	remaining: 141ms
466:	learn: 0.0071499	total: 865ms	remaining: 139ms
467:	learn: 0.0071209	total: 867ms	remaining: 137ms
468:	learn: 0.0070980	total: 868ms	remaining: 135ms
469:	learn: 0.0070960	total: 870ms	remaining: 133ms
470:	learn: 0.0070960	total: 872ms	remaining: 131ms
471:	learn: 0.0070935	total: 873ms	remaining: 130ms
472:	learn: 0.0070935	total: 875ms	remaining: 128ms
473:	learn: 0.0070900	total: 877ms	remaining: 126ms
474:	learn: 0.0070675	total: 878ms	remaining: 124ms
475:	learn: 0.0070558	total: 881ms	remaining: 122ms
476:	learn: 0.0070502	total: 882ms	remaining: 120ms
477:	learn: 0.0070244	total: 884ms	remaining: 118ms
478:	learn: 0.0070038	total: 886ms	remaining: 117ms
479:	learn: 0.0069921	total: 888ms	remaining: 115ms
480:	learn: 0.0069897	total: 889ms	remaining: 113ms
481:	learn: 0.0069787	total: 892ms	remaining: 111ms
482:	learn: 0.0069770	total: 893ms	remaining: 109ms
483:	learn: 0.0069760	total: 895ms	remaining: 107ms
484:	learn: 0.0069733	total: 897ms	remaining: 105ms
485:	learn: 0.0069714	total: 898ms	remaining: 104ms
486:	learn: 0.0069545	total: 901ms	remaining: 102ms
487:	learn: 0.0069299	total: 903ms	remaining: 99.9ms
488:	learn: 0.0069195	total: 920ms	remaining: 99.7ms
489:	learn: 0.0068961	total: 921ms	remaining: 97.8ms
490:	learn: 0.0068907	total: 923ms	remaining: 95.9ms
491:	learn: 0.0068897	total: 925ms	remaining: 94ms
492:	learn: 0.0068897	total: 926ms	remaining: 92.1ms
493:	learn: 0.0068872	total: 928ms	remaining: 90.2ms
494:	learn: 0.0068643	total: 930ms	remaining: 88.3ms
495:	learn: 0.0068525	total: 933ms	remaining: 86.5ms
496:	learn: 0.0068349	total: 935ms	remaining: 84.6ms
497:	learn: 0.0068315	total: 965ms	remaining: 85.3ms
498:	learn: 0.0068315	total: 967ms	remaining: 83.3ms
499:	learn: 0.0068307	total: 968ms	remaining: 81.3ms
500:	learn: 0.0068293	total: 970ms	remaining: 79.4ms
501:	learn: 0.0068282	total: 972ms	remaining: 77.4ms
502:	learn: 0.0068277	total: 974ms	remaining: 75.5ms
503:	learn: 0.0068264	total: 975ms	remaining: 73.5ms
504:	learn: 0.0068057	total: 977ms	remaining: 71.6ms
505:	learn: 0.0067951	total: 979ms	remaining: 69.7ms
506:	learn: 0.0067945	total: 981ms	remaining: 67.7ms
507:	learn: 0.0067932	total: 983ms	remaining: 65.8ms
508:	learn: 0.0067912	total: 985ms	remaining: 63.8ms
509:	learn: 0.0067898	total: 986ms	remaining: 61.9ms
510:	learn: 0.0067729	total: 988ms	remaining: 59.9ms
511:	learn: 0.0067637	total: 990ms	remaining: 58ms
512:	learn: 0.0067471	total: 991ms	remaining: 56ms
513:	learn: 0.0067349	total: 993ms	remaining: 54.1ms
514:	learn: 0.0067350	total: 995ms	remaining: 52.1ms
515:	learn: 0.0067350	total: 996ms	remaining: 50.2ms
516:	learn: 0.0067350	total: 998ms	remaining: 48.2ms
517:	learn: 0.0067350	total: 999ms	remaining: 46.3ms
518:	learn: 0.0067347	total: 1s	remaining: 44.4ms
519:	learn: 0.0067346	total: 1s	remaining: 42.4ms
520:	learn: 0.0067344	total: 1s	remaining: 40.5ms
521:	learn: 0.0067344	total: 1s	remaining: 38.5ms
522:	learn: 0.0067343	total: 1.01s	remaining: 36.6ms
523:	learn: 0.0067341	total: 1.01s	remaining: 34.7ms
524:	learn: 0.0067071	total: 1.01s	remaining: 32.8ms
525:	learn: 0.0067069	total: 1.01s	remaining: 30.8ms
526:	learn: 0.0067067	total: 1.01s	remaining: 28.9ms
527:	learn: 0.0067067	total: 1.02s	remaining: 26.9ms
528:	learn: 0.0067064	total: 1.02s	remaining: 25ms
529:	learn: 0.0067063	total: 1.02s	remaining: 23.1ms
530:	learn: 0.0067063	total: 1.02s	remaining: 21.2ms
531:	learn: 0.0067030	total: 1.02s	remaining: 19.2ms
532:	learn: 0.0066967	total: 1.02s	remaining: 17.3ms
533:	learn: 0.0066956	total: 1.03s	remaining: 15.4ms
534:	learn: 0.0066745	total: 1.03s	remaining: 13.5ms
535:	learn: 0.0066491	total: 1.03s	remaining: 11.5ms
536:	learn: 0.0066187	total: 1.03s	remaining: 9.61ms
537:	learn: 0.0066024	total: 1.03s	remaining: 7.69ms
538:	learn: 0.0066024	total: 1.03s	remaining: 5.76ms
539:	learn: 0.0066024	total: 1.04s	remaining: 3.84ms
540:	learn: 0.0065962	total: 1.04s	remaining: 1.92ms
541:	learn: 0.0065962	total: 1.04s	remaining: 0us
0:	learn: 0.5536936	total: 1.76ms	remaining: 954ms
1:	learn: 0.4620567	total: 3.45ms	remaining: 932ms
2:	learn: 0.3905196	total: 5.15ms	remaining: 926ms
3:	learn: 0.3521590	total: 6.84ms	remaining: 921ms
4:	learn: 0.3257945	total: 8.53ms	remaining: 916ms
5:	learn: 0.3003336	total: 10.3ms	remaining: 923ms
6:	learn: 0.2908666	total: 12.1ms	remaining: 922ms
7:	learn: 0.2787181	total: 14ms	remaining: 937ms
8:	learn: 0.2634185	total: 15.9ms	remaining: 939ms
9:	learn: 0.2518627	total: 17.6ms	remaining: 935ms
10:	learn: 0.2390562	total: 19.3ms	remaining: 932ms
11:	learn: 0.2241176	total: 21.1ms	remaining: 932ms
12:	learn: 0.2114273	total: 22.7ms	remaining: 924ms
13:	learn: 0.2013524	total: 24.2ms	remaining: 914ms
14:	learn: 0.1961105	total: 25.9ms	remaining: 911ms
15:	learn: 0.1902366	total: 27.6ms	remaining: 907ms
16:	learn: 0.1851023	total: 29.4ms	remaining: 908ms
17:	learn: 0.1796265	total: 31.1ms	remaining: 905ms
18:	learn: 0.1773016	total: 32.3ms	remaining: 889ms
19:	learn: 0.1715945	total: 33.9ms	remaining: 884ms
20:	learn: 0.1665289	total: 35.5ms	remaining: 880ms
21:	learn: 0.1593535	total: 37.1ms	remaining: 878ms
22:	learn: 0.1569057	total: 38.7ms	remaining: 873ms
23:	learn: 0.1547054	total: 40.5ms	remaining: 875ms
24:	learn: 0.1497222	total: 42.4ms	remaining: 878ms
25:	learn: 0.1435937	total: 44.2ms	remaining: 877ms
26:	learn: 0.1397433	total: 46.1ms	remaining: 879ms
27:	learn: 0.1365216	total: 47.7ms	remaining: 876ms
28:	learn: 0.1334582	total: 49.5ms	remaining: 875ms
29:	learn: 0.1305447	total: 50.9ms	remaining: 869ms
30:	learn: 0.1276142	total: 52.6ms	remaining: 867ms
31:	learn: 0.1242330	total: 54.1ms	remaining: 863ms
32:	learn: 0.1217131	total: 56.1ms	remaining: 866ms
33:	learn: 0.1180812	total: 57.8ms	remaining: 864ms
34:	learn: 0.1176416	total: 59.3ms	remaining: 860ms
35:	learn: 0.1156696	total: 61ms	remaining: 858ms
36:	learn: 0.1124585	total: 62.7ms	remaining: 856ms
37:	learn: 0.1085471	total: 64.5ms	remaining: 856ms
38:	learn: 0.1084698	total: 66.4ms	remaining: 856ms
39:	learn: 0.1072377	total: 68.5ms	remaining: 860ms
40:	learn: 0.1070357	total: 70.2ms	remaining: 858ms
41:	learn: 0.1069647	total: 72.3ms	remaining: 860ms
42:	learn: 0.1068508	total: 74.2ms	remaining: 861ms
43:	learn: 0.1068074	total: 75.7ms	remaining: 856ms
44:	learn: 0.1058743	total: 77.4ms	remaining: 855ms
45:	learn: 0.1057825	total: 79.2ms	remaining: 854ms
46:	learn: 0.1057334	total: 80.9ms	remaining: 852ms
47:	learn: 0.1056556	total: 82.5ms	remaining: 850ms
48:	learn: 0.1056012	total: 84.4ms	remaining: 849ms
49:	learn: 0.1021514	total: 86.1ms	remaining: 847ms
50:	learn: 0.0997216	total: 88ms	remaining: 847ms
51:	learn: 0.0960335	total: 89.7ms	remaining: 846ms
52:	learn: 0.0928383	total: 91.4ms	remaining: 843ms
53:	learn: 0.0899084	total: 93ms	remaining: 840ms
54:	learn: 0.0889642	total: 94.6ms	remaining: 838ms
55:	learn: 0.0862328	total: 96.2ms	remaining: 835ms
56:	learn: 0.0850831	total: 97.7ms	remaining: 832ms
57:	learn: 0.0826645	total: 99.6ms	remaining: 831ms
58:	learn: 0.0808408	total: 101ms	remaining: 830ms
59:	learn: 0.0786201	total: 104ms	remaining: 832ms
60:	learn: 0.0775009	total: 105ms	remaining: 831ms
61:	learn: 0.0759897	total: 107ms	remaining: 830ms
62:	learn: 0.0740498	total: 109ms	remaining: 828ms
63:	learn: 0.0740234	total: 111ms	remaining: 826ms
64:	learn: 0.0726077	total: 112ms	remaining: 825ms
65:	learn: 0.0711154	total: 114ms	remaining: 825ms
66:	learn: 0.0705400	total: 116ms	remaining: 823ms
67:	learn: 0.0685958	total: 118ms	remaining: 820ms
68:	learn: 0.0681650	total: 119ms	remaining: 819ms
69:	learn: 0.0669126	total: 121ms	remaining: 816ms
70:	learn: 0.0653685	total: 123ms	remaining: 814ms
71:	learn: 0.0650911	total: 124ms	remaining: 811ms
72:	learn: 0.0647249	total: 126ms	remaining: 808ms
73:	learn: 0.0631346	total: 127ms	remaining: 806ms
74:	learn: 0.0624372	total: 129ms	remaining: 805ms
75:	learn: 0.0607782	total: 131ms	remaining: 806ms
76:	learn: 0.0602196	total: 134ms	remaining: 807ms
77:	learn: 0.0592060	total: 135ms	remaining: 805ms
78:	learn: 0.0582031	total: 137ms	remaining: 803ms
79:	learn: 0.0570871	total: 139ms	remaining: 801ms
80:	learn: 0.0564353	total: 140ms	remaining: 799ms
81:	learn: 0.0551622	total: 142ms	remaining: 798ms
82:	learn: 0.0550137	total: 144ms	remaining: 797ms
83:	learn: 0.0539792	total: 146ms	remaining: 795ms
84:	learn: 0.0539671	total: 147ms	remaining: 792ms
85:	learn: 0.0536070	total: 149ms	remaining: 790ms
86:	learn: 0.0524137	total: 151ms	remaining: 789ms
87:	learn: 0.0522022	total: 152ms	remaining: 786ms
88:	learn: 0.0512385	total: 154ms	remaining: 784ms
89:	learn: 0.0512276	total: 156ms	remaining: 782ms
90:	learn: 0.0512167	total: 158ms	remaining: 781ms
91:	learn: 0.0512064	total: 159ms	remaining: 779ms
92:	learn: 0.0511961	total: 161ms	remaining: 779ms
93:	learn: 0.0511863	total: 163ms	remaining: 778ms
94:	learn: 0.0511769	total: 165ms	remaining: 777ms
95:	learn: 0.0511675	total: 167ms	remaining: 775ms
96:	learn: 0.0511464	total: 169ms	remaining: 775ms
97:	learn: 0.0511375	total: 171ms	remaining: 774ms
98:	learn: 0.0502986	total: 173ms	remaining: 773ms
99:	learn: 0.0496512	total: 174ms	remaining: 771ms
100:	learn: 0.0487529	total: 176ms	remaining: 769ms
101:	learn: 0.0487443	total: 178ms	remaining: 767ms
102:	learn: 0.0479793	total: 179ms	remaining: 764ms
103:	learn: 0.0471697	total: 181ms	remaining: 762ms
104:	learn: 0.0467649	total: 182ms	remaining: 759ms
105:	learn: 0.0462629	total: 184ms	remaining: 758ms
106:	learn: 0.0455780	total: 186ms	remaining: 755ms
107:	learn: 0.0451195	total: 188ms	remaining: 754ms
108:	learn: 0.0443222	total: 190ms	remaining: 753ms
109:	learn: 0.0437746	total: 192ms	remaining: 753ms
110:	learn: 0.0434028	total: 194ms	remaining: 753ms
111:	learn: 0.0429844	total: 197ms	remaining: 754ms
112:	learn: 0.0421089	total: 200ms	remaining: 758ms
113:	learn: 0.0416949	total: 202ms	remaining: 759ms
114:	learn: 0.0415371	total: 231ms	remaining: 857ms
115:	learn: 0.0406400	total: 234ms	remaining: 858ms
116:	learn: 0.0394225	total: 237ms	remaining: 860ms
117:	learn: 0.0392659	total: 240ms	remaining: 861ms
118:	learn: 0.0382408	total: 242ms	remaining: 862ms
119:	learn: 0.0380497	total: 245ms	remaining: 862ms
120:	learn: 0.0374845	total: 248ms	remaining: 861ms
121:	learn: 0.0370009	total: 250ms	remaining: 860ms
122:	learn: 0.0364413	total: 252ms	remaining: 858ms
123:	learn: 0.0358683	total: 254ms	remaining: 857ms
124:	learn: 0.0350320	total: 257ms	remaining: 856ms
125:	learn: 0.0350247	total: 259ms	remaining: 854ms
126:	learn: 0.0343565	total: 261ms	remaining: 852ms
127:	learn: 0.0343058	total: 263ms	remaining: 849ms
128:	learn: 0.0335507	total: 265ms	remaining: 848ms
129:	learn: 0.0335438	total: 267ms	remaining: 847ms
130:	learn: 0.0331705	total: 269ms	remaining: 845ms
131:	learn: 0.0331185	total: 271ms	remaining: 843ms
132:	learn: 0.0325609	total: 273ms	remaining: 840ms
133:	learn: 0.0325048	total: 275ms	remaining: 837ms
134:	learn: 0.0324995	total: 277ms	remaining: 834ms
135:	learn: 0.0324949	total: 279ms	remaining: 832ms
136:	learn: 0.0320643	total: 281ms	remaining: 830ms
137:	learn: 0.0317094	total: 283ms	remaining: 828ms
138:	learn: 0.0312822	total: 285ms	remaining: 825ms
139:	learn: 0.0308396	total: 286ms	remaining: 823ms
140:	learn: 0.0302161	total: 288ms	remaining: 820ms
141:	learn: 0.0301799	total: 290ms	remaining: 817ms
142:	learn: 0.0298762	total: 292ms	remaining: 814ms
143:	learn: 0.0293605	total: 293ms	remaining: 811ms
144:	learn: 0.0292258	total: 295ms	remaining: 808ms
145:	learn: 0.0287914	total: 297ms	remaining: 805ms
146:	learn: 0.0284045	total: 298ms	remaining: 802ms
147:	learn: 0.0279386	total: 300ms	remaining: 800ms
148:	learn: 0.0276206	total: 302ms	remaining: 798ms
149:	learn: 0.0272494	total: 304ms	remaining: 795ms
150:	learn: 0.0272442	total: 306ms	remaining: 792ms
151:	learn: 0.0272362	total: 308ms	remaining: 789ms
152:	learn: 0.0268076	total: 309ms	remaining: 786ms
153:	learn: 0.0264569	total: 311ms	remaining: 784ms
154:	learn: 0.0264536	total: 313ms	remaining: 780ms
155:	learn: 0.0261310	total: 315ms	remaining: 778ms
156:	learn: 0.0258491	total: 316ms	remaining: 775ms
157:	learn: 0.0258266	total: 318ms	remaining: 772ms
158:	learn: 0.0255545	total: 320ms	remaining: 770ms
159:	learn: 0.0252513	total: 322ms	remaining: 768ms
160:	learn: 0.0250485	total: 323ms	remaining: 765ms
161:	learn: 0.0247788	total: 325ms	remaining: 763ms
162:	learn: 0.0245504	total: 327ms	remaining: 761ms
163:	learn: 0.0244849	total: 329ms	remaining: 759ms
164:	learn: 0.0241818	total: 331ms	remaining: 757ms
165:	learn: 0.0240863	total: 333ms	remaining: 755ms
166:	learn: 0.0240694	total: 335ms	remaining: 753ms
167:	learn: 0.0238572	total: 338ms	remaining: 752ms
168:	learn: 0.0236944	total: 360ms	remaining: 795ms
169:	learn: 0.0235214	total: 362ms	remaining: 792ms
170:	learn: 0.0232941	total: 364ms	remaining: 789ms
171:	learn: 0.0232909	total: 366ms	remaining: 787ms
172:	learn: 0.0232887	total: 367ms	remaining: 784ms
173:	learn: 0.0232851	total: 369ms	remaining: 781ms
174:	learn: 0.0232815	total: 371ms	remaining: 778ms
175:	learn: 0.0232794	total: 373ms	remaining: 776ms
176:	learn: 0.0232770	total: 375ms	remaining: 774ms
177:	learn: 0.0230460	total: 377ms	remaining: 772ms
178:	learn: 0.0228226	total: 379ms	remaining: 769ms
179:	learn: 0.0226012	total: 381ms	remaining: 767ms
180:	learn: 0.0223245	total: 383ms	remaining: 764ms
181:	learn: 0.0223224	total: 385ms	remaining: 761ms
182:	learn: 0.0219812	total: 387ms	remaining: 759ms
183:	learn: 0.0217144	total: 388ms	remaining: 756ms
184:	learn: 0.0214536	total: 391ms	remaining: 754ms
185:	learn: 0.0212463	total: 393ms	remaining: 751ms
186:	learn: 0.0208700	total: 395ms	remaining: 749ms
187:	learn: 0.0205360	total: 396ms	remaining: 747ms
188:	learn: 0.0205358	total: 398ms	remaining: 744ms
189:	learn: 0.0202556	total: 400ms	remaining: 741ms
190:	learn: 0.0201039	total: 402ms	remaining: 738ms
191:	learn: 0.0199553	total: 404ms	remaining: 736ms
192:	learn: 0.0198702	total: 406ms	remaining: 734ms
193:	learn: 0.0198187	total: 407ms	remaining: 731ms
194:	learn: 0.0195051	total: 409ms	remaining: 729ms
195:	learn: 0.0194078	total: 411ms	remaining: 726ms
196:	learn: 0.0192400	total: 413ms	remaining: 723ms
197:	learn: 0.0190918	total: 415ms	remaining: 721ms
198:	learn: 0.0190353	total: 418ms	remaining: 720ms
199:	learn: 0.0188957	total: 420ms	remaining: 718ms
200:	learn: 0.0188124	total: 422ms	remaining: 716ms
201:	learn: 0.0186659	total: 424ms	remaining: 713ms
202:	learn: 0.0185477	total: 426ms	remaining: 711ms
203:	learn: 0.0183881	total: 428ms	remaining: 708ms
204:	learn: 0.0182658	total: 429ms	remaining: 706ms
205:	learn: 0.0181017	total: 431ms	remaining: 703ms
206:	learn: 0.0180130	total: 433ms	remaining: 700ms
207:	learn: 0.0179367	total: 434ms	remaining: 697ms
208:	learn: 0.0178625	total: 436ms	remaining: 695ms
209:	learn: 0.0177211	total: 438ms	remaining: 692ms
210:	learn: 0.0175674	total: 440ms	remaining: 690ms
211:	learn: 0.0173700	total: 441ms	remaining: 687ms
212:	learn: 0.0173284	total: 443ms	remaining: 684ms
213:	learn: 0.0171985	total: 445ms	remaining: 682ms
214:	learn: 0.0171529	total: 447ms	remaining: 680ms
215:	learn: 0.0168903	total: 449ms	remaining: 677ms
216:	learn: 0.0168054	total: 451ms	remaining: 675ms
217:	learn: 0.0166372	total: 453ms	remaining: 673ms
218:	learn: 0.0164857	total: 454ms	remaining: 670ms
219:	learn: 0.0163844	total: 460ms	remaining: 674ms
220:	learn: 0.0162182	total: 462ms	remaining: 671ms
221:	learn: 0.0161629	total: 464ms	remaining: 669ms
222:	learn: 0.0161181	total: 466ms	remaining: 667ms
223:	learn: 0.0160531	total: 468ms	remaining: 664ms
224:	learn: 0.0159789	total: 470ms	remaining: 662ms
225:	learn: 0.0158903	total: 472ms	remaining: 660ms
226:	learn: 0.0157264	total: 474ms	remaining: 657ms
227:	learn: 0.0157251	total: 475ms	remaining: 655ms
228:	learn: 0.0157245	total: 477ms	remaining: 652ms
229:	learn: 0.0157237	total: 479ms	remaining: 650ms
230:	learn: 0.0157231	total: 481ms	remaining: 647ms
231:	learn: 0.0157227	total: 482ms	remaining: 644ms
232:	learn: 0.0157223	total: 484ms	remaining: 642ms
233:	learn: 0.0157216	total: 487ms	remaining: 641ms
234:	learn: 0.0157212	total: 506ms	remaining: 661ms
235:	learn: 0.0157205	total: 509ms	remaining: 659ms
236:	learn: 0.0157203	total: 511ms	remaining: 658ms
237:	learn: 0.0157200	total: 514ms	remaining: 656ms
238:	learn: 0.0157198	total: 516ms	remaining: 655ms
239:	learn: 0.0157191	total: 519ms	remaining: 653ms
240:	learn: 0.0157188	total: 522ms	remaining: 651ms
241:	learn: 0.0157184	total: 523ms	remaining: 649ms
242:	learn: 0.0157175	total: 526ms	remaining: 647ms
243:	learn: 0.0157166	total: 527ms	remaining: 644ms
244:	learn: 0.0157162	total: 529ms	remaining: 641ms
245:	learn: 0.0157159	total: 531ms	remaining: 639ms
246:	learn: 0.0157152	total: 533ms	remaining: 637ms
247:	learn: 0.0157150	total: 535ms	remaining: 635ms
248:	learn: 0.0157149	total: 537ms	remaining: 632ms
249:	learn: 0.0157148	total: 539ms	remaining: 630ms
250:	learn: 0.0157146	total: 542ms	remaining: 628ms
251:	learn: 0.0157145	total: 544ms	remaining: 626ms
252:	learn: 0.0157142	total: 546ms	remaining: 623ms
253:	learn: 0.0157137	total: 548ms	remaining: 622ms
254:	learn: 0.0157135	total: 551ms	remaining: 620ms
255:	learn: 0.0157132	total: 556ms	remaining: 621ms
256:	learn: 0.0157130	total: 559ms	remaining: 620ms
257:	learn: 0.0157128	total: 562ms	remaining: 618ms
258:	learn: 0.0157127	total: 564ms	remaining: 617ms
259:	learn: 0.0157126	total: 567ms	remaining: 615ms
260:	learn: 0.0157125	total: 569ms	remaining: 613ms
261:	learn: 0.0157123	total: 572ms	remaining: 611ms
262:	learn: 0.0157116	total: 575ms	remaining: 610ms
263:	learn: 0.0157106	total: 578ms	remaining: 608ms
264:	learn: 0.0157095	total: 582ms	remaining: 608ms
265:	learn: 0.0157092	total: 584ms	remaining: 606ms
266:	learn: 0.0157087	total: 586ms	remaining: 604ms
267:	learn: 0.0157084	total: 605ms	remaining: 618ms
268:	learn: 0.0157078	total: 607ms	remaining: 616ms
269:	learn: 0.0157075	total: 609ms	remaining: 614ms
270:	learn: 0.0157074	total: 611ms	remaining: 611ms
271:	learn: 0.0157069	total: 613ms	remaining: 609ms
272:	learn: 0.0157069	total: 615ms	remaining: 606ms
273:	learn: 0.0157068	total: 617ms	remaining: 604ms
274:	learn: 0.0157068	total: 619ms	remaining: 601ms
275:	learn: 0.0157066	total: 621ms	remaining: 599ms
276:	learn: 0.0157064	total: 623ms	remaining: 596ms
277:	learn: 0.0157060	total: 625ms	remaining: 593ms
278:	learn: 0.0157057	total: 626ms	remaining: 590ms
279:	learn: 0.0157054	total: 629ms	remaining: 588ms
280:	learn: 0.0157052	total: 631ms	remaining: 586ms
281:	learn: 0.0157051	total: 633ms	remaining: 583ms
282:	learn: 0.0157050	total: 634ms	remaining: 581ms
283:	learn: 0.0157048	total: 636ms	remaining: 578ms
284:	learn: 0.0157047	total: 639ms	remaining: 576ms
285:	learn: 0.0157046	total: 640ms	remaining: 573ms
286:	learn: 0.0157046	total: 642ms	remaining: 571ms
287:	learn: 0.0157046	total: 644ms	remaining: 568ms
288:	learn: 0.0157046	total: 647ms	remaining: 566ms
289:	learn: 0.0157046	total: 648ms	remaining: 564ms
290:	learn: 0.0157046	total: 650ms	remaining: 561ms
291:	learn: 0.0157046	total: 651ms	remaining: 558ms
292:	learn: 0.0157046	total: 653ms	remaining: 555ms
293:	learn: 0.0157046	total: 655ms	remaining: 553ms
294:	learn: 0.0157046	total: 658ms	remaining: 551ms
295:	learn: 0.0157046	total: 660ms	remaining: 548ms
296:	learn: 0.0157046	total: 662ms	remaining: 546ms
297:	learn: 0.0157046	total: 664ms	remaining: 543ms
298:	learn: 0.0157046	total: 666ms	remaining: 541ms
299:	learn: 0.0157046	total: 668ms	remaining: 539ms
300:	learn: 0.0157046	total: 670ms	remaining: 536ms
301:	learn: 0.0157046	total: 672ms	remaining: 534ms
302:	learn: 0.0157046	total: 674ms	remaining: 532ms
303:	learn: 0.0157046	total: 676ms	remaining: 530ms
304:	learn: 0.0157046	total: 678ms	remaining: 527ms
305:	learn: 0.0157046	total: 680ms	remaining: 524ms
306:	learn: 0.0157046	total: 682ms	remaining: 522ms
307:	learn: 0.0157046	total: 684ms	remaining: 519ms
308:	learn: 0.0157046	total: 686ms	remaining: 517ms
309:	learn: 0.0157046	total: 687ms	remaining: 514ms
310:	learn: 0.0157046	total: 689ms	remaining: 512ms
311:	learn: 0.0157046	total: 691ms	remaining: 509ms
312:	learn: 0.0157046	total: 693ms	remaining: 507ms
313:	learn: 0.0157046	total: 695ms	remaining: 505ms
314:	learn: 0.0157046	total: 698ms	remaining: 503ms
315:	learn: 0.0157046	total: 699ms	remaining: 500ms
316:	learn: 0.0157046	total: 702ms	remaining: 498ms
317:	learn: 0.0157046	total: 704ms	remaining: 496ms
318:	learn: 0.0157046	total: 706ms	remaining: 493ms
319:	learn: 0.0157046	total: 707ms	remaining: 491ms
320:	learn: 0.0157046	total: 709ms	remaining: 488ms
321:	learn: 0.0157046	total: 715ms	remaining: 489ms
322:	learn: 0.0157046	total: 717ms	remaining: 486ms
323:	learn: 0.0157046	total: 719ms	remaining: 484ms
324:	learn: 0.0157046	total: 721ms	remaining: 481ms
325:	learn: 0.0157046	total: 722ms	remaining: 478ms
326:	learn: 0.0157046	total: 723ms	remaining: 476ms
327:	learn: 0.0157046	total: 725ms	remaining: 473ms
328:	learn: 0.0157046	total: 726ms	remaining: 470ms
329:	learn: 0.0157046	total: 728ms	remaining: 468ms
330:	learn: 0.0157046	total: 730ms	remaining: 466ms
331:	learn: 0.0157046	total: 732ms	remaining: 463ms
332:	learn: 0.0157046	total: 734ms	remaining: 461ms
333:	learn: 0.0157046	total: 736ms	remaining: 458ms
334:	learn: 0.0157046	total: 738ms	remaining: 456ms
335:	learn: 0.0157046	total: 740ms	remaining: 454ms
336:	learn: 0.0157046	total: 742ms	remaining: 451ms
337:	learn: 0.0157046	total: 744ms	remaining: 449ms
338:	learn: 0.0157046	total: 746ms	remaining: 446ms
339:	learn: 0.0157046	total: 747ms	remaining: 444ms
340:	learn: 0.0157046	total: 749ms	remaining: 442ms
341:	learn: 0.0157046	total: 751ms	remaining: 439ms
342:	learn: 0.0157046	total: 753ms	remaining: 437ms
343:	learn: 0.0157046	total: 754ms	remaining: 434ms
344:	learn: 0.0157046	total: 756ms	remaining: 432ms
345:	learn: 0.0157046	total: 758ms	remaining: 429ms
346:	learn: 0.0157046	total: 760ms	remaining: 427ms
347:	learn: 0.0157046	total: 762ms	remaining: 425ms
348:	learn: 0.0157046	total: 763ms	remaining: 422ms
349:	learn: 0.0157046	total: 765ms	remaining: 420ms
350:	learn: 0.0157046	total: 767ms	remaining: 417ms
351:	learn: 0.0157046	total: 769ms	remaining: 415ms
352:	learn: 0.0157046	total: 770ms	remaining: 412ms
353:	learn: 0.0157046	total: 772ms	remaining: 410ms
354:	learn: 0.0157046	total: 773ms	remaining: 407ms
355:	learn: 0.0157046	total: 775ms	remaining: 405ms
356:	learn: 0.0157046	total: 777ms	remaining: 402ms
357:	learn: 0.0157046	total: 778ms	remaining: 400ms
358:	learn: 0.0157046	total: 780ms	remaining: 398ms
359:	learn: 0.0157046	total: 782ms	remaining: 395ms
360:	learn: 0.0157046	total: 783ms	remaining: 393ms
361:	learn: 0.0157046	total: 786ms	remaining: 391ms
362:	learn: 0.0157046	total: 788ms	remaining: 389ms
363:	learn: 0.0157046	total: 790ms	remaining: 386ms
364:	learn: 0.0157046	total: 792ms	remaining: 384ms
365:	learn: 0.0157046	total: 794ms	remaining: 382ms
366:	learn: 0.0157046	total: 796ms	remaining: 380ms
367:	learn: 0.0157046	total: 798ms	remaining: 377ms
368:	learn: 0.0157046	total: 800ms	remaining: 375ms
369:	learn: 0.0157046	total: 802ms	remaining: 373ms
370:	learn: 0.0157046	total: 804ms	remaining: 370ms
371:	learn: 0.0157046	total: 805ms	remaining: 368ms
372:	learn: 0.0157046	total: 807ms	remaining: 366ms
373:	learn: 0.0157046	total: 809ms	remaining: 363ms
374:	learn: 0.0157046	total: 811ms	remaining: 361ms
375:	learn: 0.0157046	total: 813ms	remaining: 359ms
376:	learn: 0.0157046	total: 815ms	remaining: 357ms
377:	learn: 0.0157046	total: 816ms	remaining: 354ms
378:	learn: 0.0157046	total: 819ms	remaining: 352ms
379:	learn: 0.0157046	total: 821ms	remaining: 350ms
380:	learn: 0.0157046	total: 823ms	remaining: 348ms
381:	learn: 0.0157046	total: 825ms	remaining: 345ms
382:	learn: 0.0157046	total: 827ms	remaining: 343ms
383:	learn: 0.0157046	total: 828ms	remaining: 341ms
384:	learn: 0.0157046	total: 830ms	remaining: 338ms
385:	learn: 0.0157046	total: 832ms	remaining: 336ms
386:	learn: 0.0157046	total: 834ms	remaining: 334ms
387:	learn: 0.0157046	total: 835ms	remaining: 332ms
388:	learn: 0.0157046	total: 837ms	remaining: 329ms
389:	learn: 0.0157046	total: 839ms	remaining: 327ms
390:	learn: 0.0157046	total: 840ms	remaining: 324ms
391:	learn: 0.0157046	total: 842ms	remaining: 322ms
392:	learn: 0.0157046	total: 844ms	remaining: 320ms
393:	learn: 0.0157046	total: 846ms	remaining: 318ms
394:	learn: 0.0157046	total: 847ms	remaining: 315ms
395:	learn: 0.0157046	total: 849ms	remaining: 313ms
396:	learn: 0.0157046	total: 851ms	remaining: 311ms
397:	learn: 0.0157046	total: 853ms	remaining: 309ms
398:	learn: 0.0157046	total: 855ms	remaining: 306ms
399:	learn: 0.0157046	total: 857ms	remaining: 304ms
400:	learn: 0.0157046	total: 859ms	remaining: 302ms
401:	learn: 0.0157046	total: 861ms	remaining: 300ms
402:	learn: 0.0157046	total: 862ms	remaining: 297ms
403:	learn: 0.0157046	total: 864ms	remaining: 295ms
404:	learn: 0.0157046	total: 871ms	remaining: 295ms
405:	learn: 0.0157046	total: 873ms	remaining: 293ms
406:	learn: 0.0157046	total: 875ms	remaining: 290ms
407:	learn: 0.0157046	total: 878ms	remaining: 288ms
408:	learn: 0.0157046	total: 880ms	remaining: 286ms
409:	learn: 0.0157046	total: 881ms	remaining: 284ms
410:	learn: 0.0157046	total: 883ms	remaining: 281ms
411:	learn: 0.0157046	total: 885ms	remaining: 279ms
412:	learn: 0.0157046	total: 887ms	remaining: 277ms
413:	learn: 0.0157046	total: 889ms	remaining: 275ms
414:	learn: 0.0157046	total: 890ms	remaining: 272ms
415:	learn: 0.0157046	total: 892ms	remaining: 270ms
416:	learn: 0.0157046	total: 893ms	remaining: 268ms
417:	learn: 0.0157046	total: 895ms	remaining: 265ms
418:	learn: 0.0157046	total: 897ms	remaining: 263ms
419:	learn: 0.0157046	total: 898ms	remaining: 261ms
420:	learn: 0.0157046	total: 900ms	remaining: 259ms
421:	learn: 0.0157046	total: 901ms	remaining: 256ms
422:	learn: 0.0157046	total: 903ms	remaining: 254ms
423:	learn: 0.0157046	total: 905ms	remaining: 252ms
424:	learn: 0.0157046	total: 907ms	remaining: 250ms
425:	learn: 0.0157046	total: 909ms	remaining: 247ms
426:	learn: 0.0157046	total: 911ms	remaining: 245ms
427:	learn: 0.0157046	total: 913ms	remaining: 243ms
428:	learn: 0.0157046	total: 915ms	remaining: 241ms
429:	learn: 0.0157046	total: 917ms	remaining: 239ms
430:	learn: 0.0157046	total: 919ms	remaining: 237ms
431:	learn: 0.0157046	total: 921ms	remaining: 234ms
432:	learn: 0.0157046	total: 923ms	remaining: 232ms
433:	learn: 0.0157046	total: 956ms	remaining: 238ms
434:	learn: 0.0157046	total: 957ms	remaining: 236ms
435:	learn: 0.0157046	total: 959ms	remaining: 233ms
436:	learn: 0.0157046	total: 961ms	remaining: 231ms
437:	learn: 0.0157046	total: 963ms	remaining: 229ms
438:	learn: 0.0157046	total: 965ms	remaining: 226ms
439:	learn: 0.0157046	total: 966ms	remaining: 224ms
440:	learn: 0.0157046	total: 968ms	remaining: 222ms
441:	learn: 0.0157046	total: 970ms	remaining: 219ms
442:	learn: 0.0157046	total: 985ms	remaining: 220ms
443:	learn: 0.0157046	total: 987ms	remaining: 218ms
444:	learn: 0.0157046	total: 988ms	remaining: 215ms
445:	learn: 0.0157046	total: 990ms	remaining: 213ms
446:	learn: 0.0157046	total: 991ms	remaining: 211ms
447:	learn: 0.0157046	total: 993ms	remaining: 208ms
448:	learn: 0.0157046	total: 994ms	remaining: 206ms
449:	learn: 0.0157046	total: 996ms	remaining: 204ms
450:	learn: 0.0157046	total: 997ms	remaining: 201ms
451:	learn: 0.0157046	total: 999ms	remaining: 199ms
452:	learn: 0.0157046	total: 1s	remaining: 197ms
453:	learn: 0.0157046	total: 1s	remaining: 194ms
454:	learn: 0.0157046	total: 1s	remaining: 192ms
455:	learn: 0.0157046	total: 1.01s	remaining: 190ms
456:	learn: 0.0157046	total: 1.01s	remaining: 188ms
457:	learn: 0.0157046	total: 1.01s	remaining: 185ms
458:	learn: 0.0157046	total: 1.01s	remaining: 183ms
459:	learn: 0.0157046	total: 1.01s	remaining: 181ms
460:	learn: 0.0157046	total: 1.02s	remaining: 179ms
461:	learn: 0.0157046	total: 1.02s	remaining: 176ms
462:	learn: 0.0157046	total: 1.02s	remaining: 174ms
463:	learn: 0.0157046	total: 1.02s	remaining: 172ms
464:	learn: 0.0157046	total: 1.02s	remaining: 170ms
465:	learn: 0.0157046	total: 1.02s	remaining: 167ms
466:	learn: 0.0157046	total: 1.03s	remaining: 165ms
467:	learn: 0.0157046	total: 1.03s	remaining: 163ms
468:	learn: 0.0157046	total: 1.03s	remaining: 161ms
469:	learn: 0.0157046	total: 1.03s	remaining: 159ms
470:	learn: 0.0157046	total: 1.04s	remaining: 156ms
471:	learn: 0.0157046	total: 1.04s	remaining: 154ms
472:	learn: 0.0157046	total: 1.04s	remaining: 152ms
473:	learn: 0.0157046	total: 1.04s	remaining: 150ms
474:	learn: 0.0157046	total: 1.05s	remaining: 148ms
475:	learn: 0.0157046	total: 1.05s	remaining: 145ms
476:	learn: 0.0157046	total: 1.05s	remaining: 143ms
477:	learn: 0.0157046	total: 1.05s	remaining: 141ms
478:	learn: 0.0157046	total: 1.05s	remaining: 139ms
479:	learn: 0.0157046	total: 1.05s	remaining: 136ms
480:	learn: 0.0157046	total: 1.06s	remaining: 134ms
481:	learn: 0.0157046	total: 1.06s	remaining: 132ms
482:	learn: 0.0157046	total: 1.06s	remaining: 129ms
483:	learn: 0.0157046	total: 1.06s	remaining: 127ms
484:	learn: 0.0157046	total: 1.06s	remaining: 125ms
485:	learn: 0.0157046	total: 1.06s	remaining: 123ms
486:	learn: 0.0157046	total: 1.07s	remaining: 121ms
487:	learn: 0.0157046	total: 1.07s	remaining: 118ms
488:	learn: 0.0157046	total: 1.07s	remaining: 116ms
489:	learn: 0.0157046	total: 1.07s	remaining: 114ms
490:	learn: 0.0157046	total: 1.08s	remaining: 113ms
491:	learn: 0.0157046	total: 1.09s	remaining: 111ms
492:	learn: 0.0157046	total: 1.09s	remaining: 108ms
493:	learn: 0.0157046	total: 1.09s	remaining: 106ms
494:	learn: 0.0157046	total: 1.09s	remaining: 104ms
495:	learn: 0.0157046	total: 1.09s	remaining: 102ms
496:	learn: 0.0157046	total: 1.1s	remaining: 99.3ms
497:	learn: 0.0157046	total: 1.1s	remaining: 97ms
498:	learn: 0.0157046	total: 1.1s	remaining: 94.9ms
499:	learn: 0.0157046	total: 1.1s	remaining: 92.8ms
500:	learn: 0.0157046	total: 1.11s	remaining: 90.5ms
501:	learn: 0.0157046	total: 1.11s	remaining: 88.3ms
502:	learn: 0.0157046	total: 1.11s	remaining: 86.1ms
503:	learn: 0.0157046	total: 1.11s	remaining: 83.9ms
504:	learn: 0.0157046	total: 1.11s	remaining: 81.6ms
505:	learn: 0.0157046	total: 1.12s	remaining: 79.4ms
506:	learn: 0.0157046	total: 1.12s	remaining: 77.2ms
507:	learn: 0.0157046	total: 1.12s	remaining: 75ms
508:	learn: 0.0157046	total: 1.12s	remaining: 72.8ms
509:	learn: 0.0157046	total: 1.12s	remaining: 70.5ms
510:	learn: 0.0157046	total: 1.13s	remaining: 68.3ms
511:	learn: 0.0157046	total: 1.13s	remaining: 66.1ms
512:	learn: 0.0157046	total: 1.13s	remaining: 63.8ms
513:	learn: 0.0157046	total: 1.13s	remaining: 61.6ms
514:	learn: 0.0157046	total: 1.13s	remaining: 59.4ms
515:	learn: 0.0157046	total: 1.14s	remaining: 57.2ms
516:	learn: 0.0157046	total: 1.14s	remaining: 55ms
517:	learn: 0.0157046	total: 1.14s	remaining: 52.8ms
518:	learn: 0.0157046	total: 1.14s	remaining: 50.5ms
519:	learn: 0.0157046	total: 1.14s	remaining: 48.4ms
520:	learn: 0.0157046	total: 1.14s	remaining: 46.1ms
521:	learn: 0.0157046	total: 1.15s	remaining: 43.9ms
522:	learn: 0.0157046	total: 1.15s	remaining: 41.7ms
523:	learn: 0.0157046	total: 1.15s	remaining: 39.5ms
524:	learn: 0.0157046	total: 1.15s	remaining: 37.3ms
525:	learn: 0.0157046	total: 1.15s	remaining: 35.1ms
526:	learn: 0.0157046	total: 1.15s	remaining: 32.9ms
527:	learn: 0.0157046	total: 1.16s	remaining: 30.7ms
528:	learn: 0.0157046	total: 1.16s	remaining: 28.5ms
529:	learn: 0.0157046	total: 1.16s	remaining: 26.3ms
530:	learn: 0.0157046	total: 1.16s	remaining: 24.1ms
531:	learn: 0.0157046	total: 1.16s	remaining: 21.9ms
532:	learn: 0.0157046	total: 1.17s	remaining: 19.7ms
533:	learn: 0.0157046	total: 1.17s	remaining: 17.5ms
534:	learn: 0.0157046	total: 1.17s	remaining: 15.3ms
535:	learn: 0.0157046	total: 1.17s	remaining: 13.1ms
536:	learn: 0.0157046	total: 1.17s	remaining: 10.9ms
537:	learn: 0.0157046	total: 1.18s	remaining: 8.74ms
538:	learn: 0.0157046	total: 1.18s	remaining: 6.56ms
539:	learn: 0.0157046	total: 1.18s	remaining: 4.37ms
540:	learn: 0.0157046	total: 1.18s	remaining: 2.18ms
541:	learn: 0.0157046	total: 1.18s	remaining: 0us
0:	learn: 0.5621143	total: 2.07ms	remaining: 1.12s
1:	learn: 0.4981075	total: 3.56ms	remaining: 961ms
2:	learn: 0.4282670	total: 5.54ms	remaining: 996ms
3:	learn: 0.3770602	total: 8.09ms	remaining: 1.09s
4:	learn: 0.3427705	total: 10.5ms	remaining: 1.13s
5:	learn: 0.3158524	total: 12.5ms	remaining: 1.12s
6:	learn: 0.2870017	total: 14.5ms	remaining: 1.11s
7:	learn: 0.2691152	total: 17ms	remaining: 1.14s
8:	learn: 0.2564961	total: 19.2ms	remaining: 1.14s
9:	learn: 0.2457100	total: 21.2ms	remaining: 1.13s
10:	learn: 0.2340946	total: 23.1ms	remaining: 1.11s
11:	learn: 0.2273257	total: 25ms	remaining: 1.1s
12:	learn: 0.2204288	total: 27ms	remaining: 1.1s
13:	learn: 0.2136898	total: 29ms	remaining: 1.09s
14:	learn: 0.2050832	total: 31.2ms	remaining: 1.09s
15:	learn: 0.2009478	total: 32.9ms	remaining: 1.08s
16:	learn: 0.1967919	total: 34.9ms	remaining: 1.08s
17:	learn: 0.1932556	total: 36.5ms	remaining: 1.06s
18:	learn: 0.1871331	total: 38.4ms	remaining: 1.06s
19:	learn: 0.1817946	total: 40.2ms	remaining: 1.05s
20:	learn: 0.1758295	total: 42.2ms	remaining: 1.04s
21:	learn: 0.1733212	total: 44.2ms	remaining: 1.04s
22:	learn: 0.1698075	total: 46.5ms	remaining: 1.05s
23:	learn: 0.1653391	total: 48.2ms	remaining: 1.04s
24:	learn: 0.1617307	total: 50.6ms	remaining: 1.05s
25:	learn: 0.1579605	total: 52.6ms	remaining: 1.04s
26:	learn: 0.1544299	total: 55.1ms	remaining: 1.05s
27:	learn: 0.1500449	total: 56.9ms	remaining: 1.04s
28:	learn: 0.1479317	total: 58.7ms	remaining: 1.04s
29:	learn: 0.1409573	total: 60.3ms	remaining: 1.03s
30:	learn: 0.1375425	total: 62ms	remaining: 1.02s
31:	learn: 0.1342816	total: 63.8ms	remaining: 1.02s
32:	learn: 0.1315933	total: 65.8ms	remaining: 1.01s
33:	learn: 0.1285401	total: 67.5ms	remaining: 1.01s
34:	learn: 0.1251935	total: 69.4ms	remaining: 1s
35:	learn: 0.1218897	total: 71.6ms	remaining: 1.01s
36:	learn: 0.1197664	total: 73.9ms	remaining: 1.01s
37:	learn: 0.1192352	total: 75.8ms	remaining: 1s
38:	learn: 0.1185954	total: 77.9ms	remaining: 1s
39:	learn: 0.1138217	total: 79.8ms	remaining: 1s
40:	learn: 0.1116384	total: 82ms	remaining: 1s
41:	learn: 0.1107855	total: 84ms	remaining: 1s
42:	learn: 0.1077965	total: 86.4ms	remaining: 1s
43:	learn: 0.1058502	total: 88.2ms	remaining: 998ms
44:	learn: 0.1047171	total: 90.3ms	remaining: 997ms
45:	learn: 0.1044690	total: 92.2ms	remaining: 994ms
46:	learn: 0.1011383	total: 94.4ms	remaining: 995ms
47:	learn: 0.0993668	total: 96.4ms	remaining: 992ms
48:	learn: 0.0977029	total: 98.3ms	remaining: 989ms
49:	learn: 0.0955651	total: 100ms	remaining: 987ms
50:	learn: 0.0939307	total: 102ms	remaining: 984ms
51:	learn: 0.0930279	total: 104ms	remaining: 980ms
52:	learn: 0.0911636	total: 106ms	remaining: 980ms
53:	learn: 0.0887558	total: 108ms	remaining: 978ms
54:	learn: 0.0880766	total: 110ms	remaining: 974ms
55:	learn: 0.0863760	total: 112ms	remaining: 971ms
56:	learn: 0.0838589	total: 114ms	remaining: 967ms
57:	learn: 0.0818134	total: 115ms	remaining: 963ms
58:	learn: 0.0798406	total: 117ms	remaining: 961ms
59:	learn: 0.0787985	total: 120ms	remaining: 962ms
60:	learn: 0.0759910	total: 122ms	remaining: 964ms
61:	learn: 0.0745651	total: 126ms	remaining: 973ms
62:	learn: 0.0728024	total: 129ms	remaining: 978ms
63:	learn: 0.0710243	total: 131ms	remaining: 980ms
64:	learn: 0.0698273	total: 134ms	remaining: 985ms
65:	learn: 0.0679292	total: 137ms	remaining: 987ms
66:	learn: 0.0678552	total: 141ms	remaining: 1s
67:	learn: 0.0664682	total: 144ms	remaining: 1s
68:	learn: 0.0650745	total: 199ms	remaining: 1.36s
69:	learn: 0.0641333	total: 202ms	remaining: 1.36s
70:	learn: 0.0628670	total: 205ms	remaining: 1.36s
71:	learn: 0.0617233	total: 207ms	remaining: 1.35s
72:	learn: 0.0608342	total: 209ms	remaining: 1.34s
73:	learn: 0.0594987	total: 212ms	remaining: 1.34s
74:	learn: 0.0588947	total: 242ms	remaining: 1.51s
75:	learn: 0.0574285	total: 247ms	remaining: 1.51s
76:	learn: 0.0562192	total: 250ms	remaining: 1.51s
77:	learn: 0.0552051	total: 253ms	remaining: 1.5s
78:	learn: 0.0543492	total: 255ms	remaining: 1.5s
79:	learn: 0.0534488	total: 257ms	remaining: 1.49s
80:	learn: 0.0527620	total: 260ms	remaining: 1.48s
81:	learn: 0.0518792	total: 262ms	remaining: 1.47s
82:	learn: 0.0518214	total: 264ms	remaining: 1.46s
83:	learn: 0.0509130	total: 267ms	remaining: 1.45s
84:	learn: 0.0502542	total: 272ms	remaining: 1.46s
85:	learn: 0.0494121	total: 279ms	remaining: 1.48s
86:	learn: 0.0488483	total: 298ms	remaining: 1.56s
87:	learn: 0.0483052	total: 300ms	remaining: 1.55s
88:	learn: 0.0478900	total: 302ms	remaining: 1.54s
89:	learn: 0.0468684	total: 304ms	remaining: 1.53s
90:	learn: 0.0459949	total: 306ms	remaining: 1.52s
91:	learn: 0.0451340	total: 308ms	remaining: 1.51s
92:	learn: 0.0451142	total: 320ms	remaining: 1.55s
93:	learn: 0.0449909	total: 323ms	remaining: 1.54s
94:	learn: 0.0447363	total: 328ms	remaining: 1.54s
95:	learn: 0.0442586	total: 336ms	remaining: 1.56s
96:	learn: 0.0433779	total: 342ms	remaining: 1.57s
97:	learn: 0.0432757	total: 345ms	remaining: 1.56s
98:	learn: 0.0430815	total: 348ms	remaining: 1.56s
99:	learn: 0.0425846	total: 350ms	remaining: 1.55s
100:	learn: 0.0425085	total: 359ms	remaining: 1.57s
101:	learn: 0.0418052	total: 364ms	remaining: 1.57s
102:	learn: 0.0415488	total: 367ms	remaining: 1.56s
103:	learn: 0.0414744	total: 372ms	remaining: 1.56s
104:	learn: 0.0407193	total: 374ms	remaining: 1.56s
105:	learn: 0.0400532	total: 377ms	remaining: 1.55s
106:	learn: 0.0398964	total: 379ms	remaining: 1.54s
107:	learn: 0.0392215	total: 381ms	remaining: 1.53s
108:	learn: 0.0392077	total: 383ms	remaining: 1.52s
109:	learn: 0.0385891	total: 386ms	remaining: 1.51s
110:	learn: 0.0384697	total: 388ms	remaining: 1.51s
111:	learn: 0.0380051	total: 390ms	remaining: 1.5s
112:	learn: 0.0379673	total: 392ms	remaining: 1.49s
113:	learn: 0.0374928	total: 395ms	remaining: 1.48s
114:	learn: 0.0370929	total: 398ms	remaining: 1.48s
115:	learn: 0.0370283	total: 401ms	remaining: 1.47s
116:	learn: 0.0365893	total: 403ms	remaining: 1.46s
117:	learn: 0.0360569	total: 405ms	remaining: 1.46s
118:	learn: 0.0356770	total: 408ms	remaining: 1.45s
119:	learn: 0.0352814	total: 410ms	remaining: 1.44s
120:	learn: 0.0352196	total: 413ms	remaining: 1.44s
121:	learn: 0.0344624	total: 414ms	remaining: 1.43s
122:	learn: 0.0339909	total: 416ms	remaining: 1.42s
123:	learn: 0.0335368	total: 419ms	remaining: 1.41s
124:	learn: 0.0332013	total: 421ms	remaining: 1.4s
125:	learn: 0.0324301	total: 423ms	remaining: 1.4s
126:	learn: 0.0321857	total: 425ms	remaining: 1.39s
127:	learn: 0.0317680	total: 427ms	remaining: 1.38s
128:	learn: 0.0313961	total: 429ms	remaining: 1.37s
129:	learn: 0.0311029	total: 431ms	remaining: 1.36s
130:	learn: 0.0310538	total: 432ms	remaining: 1.36s
131:	learn: 0.0306942	total: 434ms	remaining: 1.35s
132:	learn: 0.0302379	total: 436ms	remaining: 1.34s
133:	learn: 0.0299661	total: 438ms	remaining: 1.33s
134:	learn: 0.0297964	total: 440ms	remaining: 1.32s
135:	learn: 0.0293727	total: 442ms	remaining: 1.32s
136:	learn: 0.0289430	total: 443ms	remaining: 1.31s
137:	learn: 0.0289352	total: 445ms	remaining: 1.3s
138:	learn: 0.0286147	total: 447ms	remaining: 1.29s
139:	learn: 0.0282784	total: 448ms	remaining: 1.29s
140:	learn: 0.0279549	total: 450ms	remaining: 1.28s
141:	learn: 0.0275928	total: 452ms	remaining: 1.27s
142:	learn: 0.0275548	total: 454ms	remaining: 1.27s
143:	learn: 0.0272642	total: 457ms	remaining: 1.26s
144:	learn: 0.0267877	total: 459ms	remaining: 1.26s
145:	learn: 0.0264559	total: 470ms	remaining: 1.27s
146:	learn: 0.0261028	total: 502ms	remaining: 1.35s
147:	learn: 0.0259082	total: 504ms	remaining: 1.34s
148:	learn: 0.0257313	total: 507ms	remaining: 1.34s
149:	learn: 0.0256885	total: 509ms	remaining: 1.33s
150:	learn: 0.0254333	total: 511ms	remaining: 1.32s
151:	learn: 0.0253953	total: 514ms	remaining: 1.32s
152:	learn: 0.0250054	total: 516ms	remaining: 1.31s
153:	learn: 0.0249424	total: 518ms	remaining: 1.3s
154:	learn: 0.0246161	total: 520ms	remaining: 1.3s
155:	learn: 0.0245806	total: 522ms	remaining: 1.29s
156:	learn: 0.0244452	total: 524ms	remaining: 1.28s
157:	learn: 0.0244103	total: 526ms	remaining: 1.28s
158:	learn: 0.0242898	total: 527ms	remaining: 1.27s
159:	learn: 0.0241406	total: 530ms	remaining: 1.26s
160:	learn: 0.0238160	total: 532ms	remaining: 1.26s
161:	learn: 0.0235883	total: 533ms	remaining: 1.25s
162:	learn: 0.0232611	total: 535ms	remaining: 1.24s
163:	learn: 0.0232307	total: 537ms	remaining: 1.24s
164:	learn: 0.0231250	total: 540ms	remaining: 1.23s
165:	learn: 0.0230299	total: 542ms	remaining: 1.23s
166:	learn: 0.0228729	total: 545ms	remaining: 1.22s
167:	learn: 0.0226616	total: 547ms	remaining: 1.22s
168:	learn: 0.0225133	total: 550ms	remaining: 1.21s
169:	learn: 0.0224261	total: 552ms	remaining: 1.21s
170:	learn: 0.0223753	total: 555ms	remaining: 1.2s
171:	learn: 0.0222319	total: 558ms	remaining: 1.2s
172:	learn: 0.0222029	total: 561ms	remaining: 1.2s
173:	learn: 0.0221562	total: 563ms	remaining: 1.19s
174:	learn: 0.0219382	total: 566ms	remaining: 1.19s
175:	learn: 0.0218951	total: 569ms	remaining: 1.18s
176:	learn: 0.0217474	total: 571ms	remaining: 1.18s
177:	learn: 0.0217136	total: 573ms	remaining: 1.17s
178:	learn: 0.0215814	total: 576ms	remaining: 1.17s
179:	learn: 0.0214392	total: 579ms	remaining: 1.16s
180:	learn: 0.0213967	total: 581ms	remaining: 1.16s
181:	learn: 0.0213589	total: 583ms	remaining: 1.15s
182:	learn: 0.0213340	total: 586ms	remaining: 1.15s
183:	learn: 0.0212999	total: 588ms	remaining: 1.14s
184:	learn: 0.0212663	total: 590ms	remaining: 1.14s
185:	learn: 0.0212496	total: 592ms	remaining: 1.13s
186:	learn: 0.0211090	total: 594ms	remaining: 1.13s
187:	learn: 0.0209708	total: 596ms	remaining: 1.12s
188:	learn: 0.0208606	total: 598ms	remaining: 1.12s
189:	learn: 0.0208605	total: 600ms	remaining: 1.11s
190:	learn: 0.0208605	total: 602ms	remaining: 1.11s
191:	learn: 0.0208605	total: 604ms	remaining: 1.1s
192:	learn: 0.0208599	total: 606ms	remaining: 1.09s
193:	learn: 0.0208600	total: 608ms	remaining: 1.09s
194:	learn: 0.0208600	total: 610ms	remaining: 1.08s
195:	learn: 0.0208595	total: 612ms	remaining: 1.08s
196:	learn: 0.0208595	total: 613ms	remaining: 1.07s
197:	learn: 0.0208594	total: 615ms	remaining: 1.07s
198:	learn: 0.0208593	total: 617ms	remaining: 1.06s
199:	learn: 0.0208593	total: 619ms	remaining: 1.06s
200:	learn: 0.0208582	total: 622ms	remaining: 1.05s
201:	learn: 0.0208566	total: 623ms	remaining: 1.05s
202:	learn: 0.0208564	total: 625ms	remaining: 1.04s
203:	learn: 0.0208560	total: 627ms	remaining: 1.04s
204:	learn: 0.0208559	total: 628ms	remaining: 1.03s
205:	learn: 0.0208553	total: 630ms	remaining: 1.03s
206:	learn: 0.0208554	total: 631ms	remaining: 1.02s
207:	learn: 0.0208541	total: 633ms	remaining: 1.02s
208:	learn: 0.0208541	total: 635ms	remaining: 1.01s
209:	learn: 0.0208541	total: 636ms	remaining: 1s
210:	learn: 0.0208541	total: 638ms	remaining: 1s
211:	learn: 0.0208539	total: 639ms	remaining: 995ms
212:	learn: 0.0208535	total: 641ms	remaining: 989ms
213:	learn: 0.0208535	total: 642ms	remaining: 984ms
214:	learn: 0.0208534	total: 644ms	remaining: 979ms
215:	learn: 0.0208532	total: 645ms	remaining: 974ms
216:	learn: 0.0208531	total: 647ms	remaining: 968ms
217:	learn: 0.0208532	total: 648ms	remaining: 964ms
218:	learn: 0.0208519	total: 651ms	remaining: 959ms
219:	learn: 0.0208517	total: 652ms	remaining: 955ms
220:	learn: 0.0208515	total: 654ms	remaining: 950ms
221:	learn: 0.0208513	total: 655ms	remaining: 945ms
222:	learn: 0.0208511	total: 657ms	remaining: 940ms
223:	learn: 0.0208510	total: 659ms	remaining: 935ms
224:	learn: 0.0208506	total: 660ms	remaining: 930ms
225:	learn: 0.0208505	total: 662ms	remaining: 925ms
226:	learn: 0.0208493	total: 664ms	remaining: 921ms
227:	learn: 0.0208490	total: 665ms	remaining: 916ms
228:	learn: 0.0208489	total: 667ms	remaining: 912ms
229:	learn: 0.0208486	total: 669ms	remaining: 908ms
230:	learn: 0.0208484	total: 688ms	remaining: 926ms
231:	learn: 0.0206687	total: 690ms	remaining: 922ms
232:	learn: 0.0206490	total: 691ms	remaining: 917ms
233:	learn: 0.0206245	total: 693ms	remaining: 912ms
234:	learn: 0.0205944	total: 695ms	remaining: 908ms
235:	learn: 0.0203100	total: 696ms	remaining: 903ms
236:	learn: 0.0202351	total: 698ms	remaining: 898ms
237:	learn: 0.0201340	total: 699ms	remaining: 893ms
238:	learn: 0.0199051	total: 701ms	remaining: 889ms
239:	learn: 0.0198859	total: 703ms	remaining: 884ms
240:	learn: 0.0198179	total: 704ms	remaining: 880ms
241:	learn: 0.0197285	total: 706ms	remaining: 876ms
242:	learn: 0.0196438	total: 708ms	remaining: 871ms
243:	learn: 0.0195288	total: 710ms	remaining: 867ms
244:	learn: 0.0194899	total: 712ms	remaining: 863ms
245:	learn: 0.0194735	total: 714ms	remaining: 859ms
246:	learn: 0.0194461	total: 716ms	remaining: 855ms
247:	learn: 0.0193025	total: 717ms	remaining: 850ms
248:	learn: 0.0190974	total: 719ms	remaining: 846ms
249:	learn: 0.0190940	total: 721ms	remaining: 842ms
250:	learn: 0.0190794	total: 723ms	remaining: 838ms
251:	learn: 0.0189752	total: 724ms	remaining: 834ms
252:	learn: 0.0187750	total: 726ms	remaining: 829ms
253:	learn: 0.0187640	total: 728ms	remaining: 825ms
254:	learn: 0.0187500	total: 730ms	remaining: 821ms
255:	learn: 0.0187358	total: 732ms	remaining: 818ms
256:	learn: 0.0186250	total: 734ms	remaining: 813ms
257:	learn: 0.0185768	total: 736ms	remaining: 810ms
258:	learn: 0.0183818	total: 738ms	remaining: 806ms
259:	learn: 0.0183676	total: 740ms	remaining: 802ms
260:	learn: 0.0183479	total: 742ms	remaining: 799ms
261:	learn: 0.0182660	total: 744ms	remaining: 795ms
262:	learn: 0.0181742	total: 746ms	remaining: 792ms
263:	learn: 0.0180889	total: 748ms	remaining: 788ms
264:	learn: 0.0179550	total: 751ms	remaining: 785ms
265:	learn: 0.0179110	total: 753ms	remaining: 781ms
266:	learn: 0.0178225	total: 755ms	remaining: 777ms
267:	learn: 0.0177186	total: 757ms	remaining: 773ms
268:	learn: 0.0175784	total: 758ms	remaining: 770ms
269:	learn: 0.0174032	total: 760ms	remaining: 766ms
270:	learn: 0.0173571	total: 762ms	remaining: 762ms
271:	learn: 0.0173331	total: 764ms	remaining: 758ms
272:	learn: 0.0172106	total: 765ms	remaining: 754ms
273:	learn: 0.0171688	total: 767ms	remaining: 751ms
274:	learn: 0.0171431	total: 769ms	remaining: 747ms
275:	learn: 0.0170678	total: 771ms	remaining: 743ms
276:	learn: 0.0169731	total: 773ms	remaining: 740ms
277:	learn: 0.0169578	total: 775ms	remaining: 736ms
278:	learn: 0.0169245	total: 777ms	remaining: 732ms
279:	learn: 0.0167901	total: 779ms	remaining: 729ms
280:	learn: 0.0166225	total: 781ms	remaining: 725ms
281:	learn: 0.0165309	total: 805ms	remaining: 742ms
282:	learn: 0.0163092	total: 807ms	remaining: 739ms
283:	learn: 0.0162448	total: 809ms	remaining: 735ms
284:	learn: 0.0161128	total: 811ms	remaining: 731ms
285:	learn: 0.0160749	total: 813ms	remaining: 727ms
286:	learn: 0.0159618	total: 814ms	remaining: 724ms
287:	learn: 0.0157805	total: 816ms	remaining: 720ms
288:	learn: 0.0157469	total: 818ms	remaining: 716ms
289:	learn: 0.0156526	total: 821ms	remaining: 713ms
290:	learn: 0.0156206	total: 823ms	remaining: 710ms
291:	learn: 0.0156120	total: 824ms	remaining: 706ms
292:	learn: 0.0156039	total: 826ms	remaining: 702ms
293:	learn: 0.0155948	total: 828ms	remaining: 699ms
294:	learn: 0.0155208	total: 831ms	remaining: 696ms
295:	learn: 0.0155207	total: 833ms	remaining: 692ms
296:	learn: 0.0153484	total: 835ms	remaining: 689ms
297:	learn: 0.0153471	total: 837ms	remaining: 685ms
298:	learn: 0.0153416	total: 839ms	remaining: 682ms
299:	learn: 0.0153360	total: 842ms	remaining: 679ms
300:	learn: 0.0153359	total: 843ms	remaining: 675ms
301:	learn: 0.0153356	total: 846ms	remaining: 672ms
302:	learn: 0.0153355	total: 848ms	remaining: 669ms
303:	learn: 0.0153355	total: 849ms	remaining: 665ms
304:	learn: 0.0153351	total: 851ms	remaining: 661ms
305:	learn: 0.0153350	total: 853ms	remaining: 658ms
306:	learn: 0.0153343	total: 855ms	remaining: 654ms
307:	learn: 0.0153112	total: 857ms	remaining: 651ms
308:	learn: 0.0153113	total: 859ms	remaining: 647ms
309:	learn: 0.0153110	total: 861ms	remaining: 644ms
310:	learn: 0.0153107	total: 863ms	remaining: 641ms
311:	learn: 0.0153108	total: 864ms	remaining: 637ms
312:	learn: 0.0153100	total: 866ms	remaining: 634ms
313:	learn: 0.0153098	total: 868ms	remaining: 630ms
314:	learn: 0.0153093	total: 869ms	remaining: 627ms
315:	learn: 0.0153092	total: 871ms	remaining: 623ms
316:	learn: 0.0153091	total: 873ms	remaining: 620ms
317:	learn: 0.0153085	total: 875ms	remaining: 617ms
318:	learn: 0.0153085	total: 877ms	remaining: 613ms
319:	learn: 0.0153085	total: 879ms	remaining: 610ms
320:	learn: 0.0153078	total: 881ms	remaining: 606ms
321:	learn: 0.0153072	total: 882ms	remaining: 603ms
322:	learn: 0.0153069	total: 884ms	remaining: 600ms
323:	learn: 0.0153071	total: 886ms	remaining: 596ms
324:	learn: 0.0153068	total: 888ms	remaining: 593ms
325:	learn: 0.0153066	total: 889ms	remaining: 589ms
326:	learn: 0.0153064	total: 904ms	remaining: 595ms
327:	learn: 0.0153060	total: 906ms	remaining: 591ms
328:	learn: 0.0153056	total: 908ms	remaining: 588ms
329:	learn: 0.0153049	total: 910ms	remaining: 584ms
330:	learn: 0.0153050	total: 911ms	remaining: 581ms
331:	learn: 0.0153047	total: 913ms	remaining: 578ms
332:	learn: 0.0153040	total: 915ms	remaining: 574ms
333:	learn: 0.0153040	total: 917ms	remaining: 571ms
334:	learn: 0.0153032	total: 918ms	remaining: 567ms
335:	learn: 0.0153029	total: 920ms	remaining: 564ms
336:	learn: 0.0153028	total: 922ms	remaining: 561ms
337:	learn: 0.0153029	total: 924ms	remaining: 558ms
338:	learn: 0.0153029	total: 926ms	remaining: 554ms
339:	learn: 0.0153022	total: 928ms	remaining: 551ms
340:	learn: 0.0151995	total: 930ms	remaining: 548ms
341:	learn: 0.0150450	total: 932ms	remaining: 545ms
342:	learn: 0.0149108	total: 934ms	remaining: 542ms
343:	learn: 0.0149108	total: 935ms	remaining: 538ms
344:	learn: 0.0149106	total: 963ms	remaining: 550ms
345:	learn: 0.0149105	total: 964ms	remaining: 546ms
346:	learn: 0.0149104	total: 966ms	remaining: 543ms
347:	learn: 0.0149102	total: 967ms	remaining: 539ms
348:	learn: 0.0149103	total: 969ms	remaining: 536ms
349:	learn: 0.0149101	total: 971ms	remaining: 532ms
350:	learn: 0.0149101	total: 972ms	remaining: 529ms
351:	learn: 0.0149101	total: 974ms	remaining: 526ms
352:	learn: 0.0149101	total: 976ms	remaining: 522ms
353:	learn: 0.0149102	total: 977ms	remaining: 519ms
354:	learn: 0.0149100	total: 979ms	remaining: 516ms
355:	learn: 0.0149100	total: 981ms	remaining: 513ms
356:	learn: 0.0149099	total: 983ms	remaining: 509ms
357:	learn: 0.0149098	total: 984ms	remaining: 506ms
358:	learn: 0.0149098	total: 986ms	remaining: 503ms
359:	learn: 0.0149097	total: 988ms	remaining: 499ms
360:	learn: 0.0149097	total: 989ms	remaining: 496ms
361:	learn: 0.0149097	total: 991ms	remaining: 493ms
362:	learn: 0.0149096	total: 993ms	remaining: 489ms
363:	learn: 0.0149095	total: 995ms	remaining: 486ms
364:	learn: 0.0149095	total: 997ms	remaining: 483ms
365:	learn: 0.0149094	total: 1.02s	remaining: 491ms
366:	learn: 0.0149094	total: 1.02s	remaining: 488ms
367:	learn: 0.0149087	total: 1.02s	remaining: 484ms
368:	learn: 0.0149085	total: 1.03s	remaining: 481ms
369:	learn: 0.0149086	total: 1.03s	remaining: 478ms
370:	learn: 0.0149085	total: 1.03s	remaining: 475ms
371:	learn: 0.0149084	total: 1.03s	remaining: 472ms
372:	learn: 0.0149084	total: 1.03s	remaining: 469ms
373:	learn: 0.0149083	total: 1.03s	remaining: 465ms
374:	learn: 0.0149083	total: 1.04s	remaining: 462ms
375:	learn: 0.0149082	total: 1.04s	remaining: 459ms
376:	learn: 0.0149081	total: 1.04s	remaining: 456ms
377:	learn: 0.0149070	total: 1.04s	remaining: 452ms
378:	learn: 0.0149070	total: 1.04s	remaining: 449ms
379:	learn: 0.0149070	total: 1.04s	remaining: 446ms
380:	learn: 0.0149070	total: 1.05s	remaining: 443ms
381:	learn: 0.0149069	total: 1.05s	remaining: 439ms
382:	learn: 0.0149068	total: 1.05s	remaining: 436ms
383:	learn: 0.0149069	total: 1.05s	remaining: 433ms
384:	learn: 0.0149068	total: 1.05s	remaining: 430ms
385:	learn: 0.0149065	total: 1.06s	remaining: 427ms
386:	learn: 0.0149064	total: 1.06s	remaining: 424ms
387:	learn: 0.0149064	total: 1.06s	remaining: 421ms
388:	learn: 0.0149064	total: 1.06s	remaining: 418ms
389:	learn: 0.0149063	total: 1.06s	remaining: 415ms
390:	learn: 0.0149063	total: 1.06s	remaining: 411ms
391:	learn: 0.0149062	total: 1.07s	remaining: 408ms
392:	learn: 0.0149061	total: 1.07s	remaining: 405ms
393:	learn: 0.0149059	total: 1.07s	remaining: 402ms
394:	learn: 0.0148991	total: 1.07s	remaining: 399ms
395:	learn: 0.0147978	total: 1.07s	remaining: 396ms
396:	learn: 0.0147698	total: 1.08s	remaining: 393ms
397:	learn: 0.0147573	total: 1.08s	remaining: 390ms
398:	learn: 0.0146881	total: 1.08s	remaining: 387ms
399:	learn: 0.0145933	total: 1.08s	remaining: 384ms
400:	learn: 0.0145784	total: 1.08s	remaining: 381ms
401:	learn: 0.0144764	total: 1.08s	remaining: 378ms
402:	learn: 0.0144519	total: 1.09s	remaining: 375ms
403:	learn: 0.0143117	total: 1.09s	remaining: 372ms
404:	learn: 0.0142452	total: 1.09s	remaining: 369ms
405:	learn: 0.0142126	total: 1.09s	remaining: 366ms
406:	learn: 0.0140686	total: 1.09s	remaining: 363ms
407:	learn: 0.0139228	total: 1.1s	remaining: 360ms
408:	learn: 0.0139227	total: 1.1s	remaining: 357ms
409:	learn: 0.0138407	total: 1.1s	remaining: 354ms
410:	learn: 0.0138407	total: 1.1s	remaining: 351ms
411:	learn: 0.0137825	total: 1.1s	remaining: 348ms
412:	learn: 0.0137794	total: 1.1s	remaining: 345ms
413:	learn: 0.0137513	total: 1.11s	remaining: 342ms
414:	learn: 0.0137454	total: 1.11s	remaining: 339ms
415:	learn: 0.0136720	total: 1.11s	remaining: 336ms
416:	learn: 0.0136179	total: 1.11s	remaining: 333ms
417:	learn: 0.0135372	total: 1.11s	remaining: 330ms
418:	learn: 0.0134194	total: 1.11s	remaining: 328ms
419:	learn: 0.0133688	total: 1.12s	remaining: 325ms
420:	learn: 0.0133100	total: 1.14s	remaining: 326ms
421:	learn: 0.0132628	total: 1.14s	remaining: 323ms
422:	learn: 0.0131895	total: 1.14s	remaining: 321ms
423:	learn: 0.0130985	total: 1.14s	remaining: 318ms
424:	learn: 0.0129772	total: 1.14s	remaining: 315ms
425:	learn: 0.0128793	total: 1.14s	remaining: 312ms
426:	learn: 0.0128263	total: 1.15s	remaining: 309ms
427:	learn: 0.0128260	total: 1.15s	remaining: 306ms
428:	learn: 0.0128002	total: 1.15s	remaining: 303ms
429:	learn: 0.0127708	total: 1.17s	remaining: 304ms
430:	learn: 0.0127482	total: 1.17s	remaining: 301ms
431:	learn: 0.0127272	total: 1.17s	remaining: 298ms
432:	learn: 0.0126150	total: 1.17s	remaining: 295ms
433:	learn: 0.0124747	total: 1.17s	remaining: 292ms
434:	learn: 0.0123695	total: 1.17s	remaining: 289ms
435:	learn: 0.0123294	total: 1.18s	remaining: 286ms
436:	learn: 0.0123166	total: 1.18s	remaining: 283ms
437:	learn: 0.0122697	total: 1.18s	remaining: 280ms
438:	learn: 0.0121847	total: 1.18s	remaining: 277ms
439:	learn: 0.0121722	total: 1.18s	remaining: 274ms
440:	learn: 0.0120988	total: 1.18s	remaining: 271ms
441:	learn: 0.0120318	total: 1.19s	remaining: 268ms
442:	learn: 0.0120158	total: 1.19s	remaining: 265ms
443:	learn: 0.0119322	total: 1.19s	remaining: 262ms
444:	learn: 0.0118705	total: 1.19s	remaining: 260ms
445:	learn: 0.0118262	total: 1.19s	remaining: 257ms
446:	learn: 0.0118063	total: 1.19s	remaining: 254ms
447:	learn: 0.0117458	total: 1.2s	remaining: 251ms
448:	learn: 0.0117416	total: 1.2s	remaining: 248ms
449:	learn: 0.0117233	total: 1.2s	remaining: 245ms
450:	learn: 0.0116609	total: 1.2s	remaining: 242ms
451:	learn: 0.0116439	total: 1.2s	remaining: 239ms
452:	learn: 0.0116381	total: 1.2s	remaining: 236ms
453:	learn: 0.0115504	total: 1.2s	remaining: 234ms
454:	learn: 0.0115053	total: 1.21s	remaining: 231ms
455:	learn: 0.0115020	total: 1.21s	remaining: 228ms
456:	learn: 0.0114366	total: 1.21s	remaining: 225ms
457:	learn: 0.0113615	total: 1.21s	remaining: 222ms
458:	learn: 0.0112818	total: 1.21s	remaining: 220ms
459:	learn: 0.0112815	total: 1.22s	remaining: 217ms
460:	learn: 0.0112743	total: 1.22s	remaining: 214ms
461:	learn: 0.0112693	total: 1.22s	remaining: 211ms
462:	learn: 0.0112300	total: 1.22s	remaining: 209ms
463:	learn: 0.0112298	total: 1.22s	remaining: 206ms
464:	learn: 0.0111516	total: 1.23s	remaining: 203ms
465:	learn: 0.0110816	total: 1.23s	remaining: 200ms
466:	learn: 0.0109942	total: 1.23s	remaining: 198ms
467:	learn: 0.0109888	total: 1.23s	remaining: 195ms
468:	learn: 0.0109562	total: 1.23s	remaining: 192ms
469:	learn: 0.0109048	total: 1.24s	remaining: 189ms
470:	learn: 0.0108810	total: 1.24s	remaining: 187ms
471:	learn: 0.0108809	total: 1.24s	remaining: 184ms
472:	learn: 0.0108124	total: 1.24s	remaining: 181ms
473:	learn: 0.0108060	total: 1.24s	remaining: 178ms
474:	learn: 0.0107175	total: 1.25s	remaining: 176ms
475:	learn: 0.0106683	total: 1.25s	remaining: 173ms
476:	learn: 0.0106443	total: 1.25s	remaining: 170ms
477:	learn: 0.0105757	total: 1.25s	remaining: 168ms
478:	learn: 0.0105188	total: 1.25s	remaining: 165ms
479:	learn: 0.0105140	total: 1.25s	remaining: 162ms
480:	learn: 0.0105047	total: 1.26s	remaining: 160ms
481:	learn: 0.0104993	total: 1.26s	remaining: 157ms
482:	learn: 0.0104661	total: 1.26s	remaining: 154ms
483:	learn: 0.0103940	total: 1.26s	remaining: 151ms
484:	learn: 0.0103884	total: 1.26s	remaining: 149ms
485:	learn: 0.0103352	total: 1.27s	remaining: 146ms
486:	learn: 0.0103205	total: 1.27s	remaining: 143ms
487:	learn: 0.0102937	total: 1.27s	remaining: 141ms
488:	learn: 0.0102881	total: 1.27s	remaining: 138ms
489:	learn: 0.0102373	total: 1.28s	remaining: 135ms
490:	learn: 0.0102232	total: 1.28s	remaining: 133ms
491:	learn: 0.0102150	total: 1.28s	remaining: 130ms
492:	learn: 0.0101570	total: 1.28s	remaining: 127ms
493:	learn: 0.0101524	total: 1.28s	remaining: 125ms
494:	learn: 0.0100712	total: 1.28s	remaining: 122ms
495:	learn: 0.0100659	total: 1.29s	remaining: 119ms
496:	learn: 0.0100390	total: 1.29s	remaining: 117ms
497:	learn: 0.0100327	total: 1.29s	remaining: 114ms
498:	learn: 0.0099897	total: 1.29s	remaining: 111ms
499:	learn: 0.0099788	total: 1.29s	remaining: 109ms
500:	learn: 0.0099508	total: 1.3s	remaining: 106ms
501:	learn: 0.0099486	total: 1.3s	remaining: 104ms
502:	learn: 0.0099130	total: 1.3s	remaining: 101ms
503:	learn: 0.0099092	total: 1.3s	remaining: 98.3ms
504:	learn: 0.0098830	total: 1.3s	remaining: 95.7ms
505:	learn: 0.0098583	total: 1.31s	remaining: 93ms
506:	learn: 0.0098578	total: 1.31s	remaining: 90.4ms
507:	learn: 0.0098533	total: 1.31s	remaining: 87.8ms
508:	learn: 0.0098444	total: 1.31s	remaining: 85.2ms
509:	learn: 0.0098159	total: 1.31s	remaining: 82.5ms
510:	learn: 0.0097607	total: 1.32s	remaining: 79.9ms
511:	learn: 0.0097098	total: 1.32s	remaining: 77.3ms
512:	learn: 0.0097055	total: 1.32s	remaining: 74.7ms
513:	learn: 0.0096485	total: 1.32s	remaining: 72.1ms
514:	learn: 0.0096460	total: 1.32s	remaining: 69.5ms
515:	learn: 0.0096044	total: 1.33s	remaining: 66.9ms
516:	learn: 0.0095546	total: 1.33s	remaining: 64.3ms
517:	learn: 0.0095142	total: 1.33s	remaining: 61.7ms
518:	learn: 0.0095130	total: 1.33s	remaining: 59.1ms
519:	learn: 0.0094712	total: 1.33s	remaining: 56.5ms
520:	learn: 0.0094227	total: 1.34s	remaining: 53.9ms
521:	learn: 0.0093943	total: 1.34s	remaining: 51.3ms
522:	learn: 0.0093567	total: 1.34s	remaining: 48.7ms
523:	learn: 0.0093204	total: 1.34s	remaining: 46.1ms
524:	learn: 0.0093030	total: 1.34s	remaining: 43.5ms
525:	learn: 0.0092996	total: 1.35s	remaining: 40.9ms
526:	learn: 0.0092632	total: 1.35s	remaining: 38.4ms
527:	learn: 0.0092632	total: 1.35s	remaining: 35.8ms
528:	learn: 0.0092210	total: 1.35s	remaining: 33.2ms
529:	learn: 0.0092164	total: 1.35s	remaining: 30.6ms
530:	learn: 0.0092034	total: 1.35s	remaining: 28.1ms
531:	learn: 0.0091921	total: 1.36s	remaining: 25.5ms
532:	learn: 0.0091468	total: 1.36s	remaining: 22.9ms
533:	learn: 0.0091250	total: 1.36s	remaining: 20.4ms
534:	learn: 0.0090986	total: 1.36s	remaining: 17.8ms
535:	learn: 0.0090900	total: 1.36s	remaining: 15.3ms
536:	learn: 0.0090596	total: 1.36s	remaining: 12.7ms
537:	learn: 0.0090215	total: 1.37s	remaining: 10.2ms
538:	learn: 0.0089790	total: 1.37s	remaining: 7.62ms
539:	learn: 0.0089642	total: 1.37s	remaining: 5.08ms
540:	learn: 0.0089403	total: 1.37s	remaining: 2.54ms
541:	learn: 0.0089064	total: 1.38s	remaining: 0us
0:	learn: 0.5462232	total: 1.94ms	remaining: 1.05s
1:	learn: 0.4481282	total: 3.81ms	remaining: 1.03s
2:	learn: 0.4223634	total: 4.89ms	remaining: 879ms
3:	learn: 0.3878420	total: 6.54ms	remaining: 880ms
4:	learn: 0.3492731	total: 8.21ms	remaining: 882ms
5:	learn: 0.3169058	total: 10.2ms	remaining: 911ms
6:	learn: 0.3063602	total: 12ms	remaining: 918ms
7:	learn: 0.2912522	total: 13.8ms	remaining: 921ms
8:	learn: 0.2784915	total: 15.8ms	remaining: 937ms
9:	learn: 0.2680335	total: 18.1ms	remaining: 961ms
10:	learn: 0.2621001	total: 20.5ms	remaining: 990ms
11:	learn: 0.2466462	total: 22.4ms	remaining: 989ms
12:	learn: 0.2378492	total: 24ms	remaining: 978ms
13:	learn: 0.2278061	total: 26ms	remaining: 979ms
14:	learn: 0.2256873	total: 35.9ms	remaining: 1.26s
15:	learn: 0.2126807	total: 37.9ms	remaining: 1.25s
16:	learn: 0.2093381	total: 39.9ms	remaining: 1.23s
17:	learn: 0.2067944	total: 41.8ms	remaining: 1.22s
18:	learn: 0.2008839	total: 43.8ms	remaining: 1.2s
19:	learn: 0.1959558	total: 46ms	remaining: 1.2s
20:	learn: 0.1912118	total: 47.8ms	remaining: 1.19s
21:	learn: 0.1876611	total: 49.7ms	remaining: 1.17s
22:	learn: 0.1800414	total: 51.2ms	remaining: 1.16s
23:	learn: 0.1759717	total: 52.9ms	remaining: 1.14s
24:	learn: 0.1688134	total: 54.7ms	remaining: 1.13s
25:	learn: 0.1645282	total: 56.4ms	remaining: 1.12s
26:	learn: 0.1606187	total: 58.5ms	remaining: 1.12s
27:	learn: 0.1561130	total: 60.6ms	remaining: 1.11s
28:	learn: 0.1511941	total: 62.4ms	remaining: 1.1s
29:	learn: 0.1477518	total: 64.1ms	remaining: 1.09s
30:	learn: 0.1428439	total: 66.1ms	remaining: 1.09s
31:	learn: 0.1375433	total: 67.9ms	remaining: 1.08s
32:	learn: 0.1343689	total: 69.9ms	remaining: 1.08s
33:	learn: 0.1310705	total: 72.2ms	remaining: 1.08s
34:	learn: 0.1234787	total: 74.8ms	remaining: 1.08s
35:	learn: 0.1210803	total: 77.2ms	remaining: 1.08s
36:	learn: 0.1206000	total: 79.3ms	remaining: 1.08s
37:	learn: 0.1185994	total: 81.5ms	remaining: 1.08s
38:	learn: 0.1168147	total: 83.8ms	remaining: 1.08s
39:	learn: 0.1153836	total: 85.8ms	remaining: 1.08s
40:	learn: 0.1118723	total: 87.8ms	remaining: 1.07s
41:	learn: 0.1087689	total: 90.5ms	remaining: 1.08s
42:	learn: 0.1053223	total: 93.3ms	remaining: 1.08s
43:	learn: 0.1022122	total: 95.2ms	remaining: 1.08s
44:	learn: 0.1003471	total: 97.6ms	remaining: 1.08s
45:	learn: 0.0984609	total: 99.8ms	remaining: 1.08s
46:	learn: 0.0956185	total: 102ms	remaining: 1.08s
47:	learn: 0.0915527	total: 106ms	remaining: 1.09s
48:	learn: 0.0885963	total: 109ms	remaining: 1.1s
49:	learn: 0.0873073	total: 112ms	remaining: 1.1s
50:	learn: 0.0853496	total: 115ms	remaining: 1.1s
51:	learn: 0.0835771	total: 117ms	remaining: 1.1s
52:	learn: 0.0820201	total: 120ms	remaining: 1.1s
53:	learn: 0.0800876	total: 123ms	remaining: 1.11s
54:	learn: 0.0798378	total: 126ms	remaining: 1.11s
55:	learn: 0.0792381	total: 129ms	remaining: 1.11s
56:	learn: 0.0783879	total: 131ms	remaining: 1.12s
57:	learn: 0.0782159	total: 134ms	remaining: 1.12s
58:	learn: 0.0778676	total: 137ms	remaining: 1.12s
59:	learn: 0.0751589	total: 140ms	remaining: 1.13s
60:	learn: 0.0741868	total: 143ms	remaining: 1.13s
61:	learn: 0.0735434	total: 145ms	remaining: 1.13s
62:	learn: 0.0720811	total: 148ms	remaining: 1.12s
63:	learn: 0.0702709	total: 150ms	remaining: 1.12s
64:	learn: 0.0690201	total: 152ms	remaining: 1.11s
65:	learn: 0.0675258	total: 154ms	remaining: 1.11s
66:	learn: 0.0673166	total: 156ms	remaining: 1.11s
67:	learn: 0.0663634	total: 159ms	remaining: 1.11s
68:	learn: 0.0650936	total: 161ms	remaining: 1.1s
69:	learn: 0.0635859	total: 164ms	remaining: 1.11s
70:	learn: 0.0618393	total: 167ms	remaining: 1.11s
71:	learn: 0.0603921	total: 169ms	remaining: 1.1s
72:	learn: 0.0587403	total: 172ms	remaining: 1.1s
73:	learn: 0.0583785	total: 175ms	remaining: 1.1s
74:	learn: 0.0576101	total: 178ms	remaining: 1.11s
75:	learn: 0.0564366	total: 194ms	remaining: 1.19s
76:	learn: 0.0549463	total: 196ms	remaining: 1.19s
77:	learn: 0.0533797	total: 199ms	remaining: 1.18s
78:	learn: 0.0519224	total: 201ms	remaining: 1.18s
79:	learn: 0.0513485	total: 203ms	remaining: 1.17s
80:	learn: 0.0500414	total: 205ms	remaining: 1.17s
81:	learn: 0.0496808	total: 208ms	remaining: 1.17s
82:	learn: 0.0485737	total: 211ms	remaining: 1.17s
83:	learn: 0.0477186	total: 214ms	remaining: 1.17s
84:	learn: 0.0464761	total: 217ms	remaining: 1.17s
85:	learn: 0.0457213	total: 221ms	remaining: 1.17s
86:	learn: 0.0451813	total: 223ms	remaining: 1.17s
87:	learn: 0.0443242	total: 226ms	remaining: 1.16s
88:	learn: 0.0435829	total: 228ms	remaining: 1.16s
89:	learn: 0.0435107	total: 230ms	remaining: 1.15s
90:	learn: 0.0431989	total: 232ms	remaining: 1.15s
91:	learn: 0.0423142	total: 234ms	remaining: 1.15s
92:	learn: 0.0414569	total: 237ms	remaining: 1.15s
93:	learn: 0.0409319	total: 239ms	remaining: 1.14s
94:	learn: 0.0403143	total: 241ms	remaining: 1.14s
95:	learn: 0.0399690	total: 244ms	remaining: 1.13s
96:	learn: 0.0398705	total: 246ms	remaining: 1.13s
97:	learn: 0.0396400	total: 249ms	remaining: 1.13s
98:	learn: 0.0394480	total: 252ms	remaining: 1.13s
99:	learn: 0.0391374	total: 255ms	remaining: 1.13s
100:	learn: 0.0388782	total: 257ms	remaining: 1.12s
101:	learn: 0.0382279	total: 259ms	remaining: 1.12s
102:	learn: 0.0379604	total: 261ms	remaining: 1.11s
103:	learn: 0.0375874	total: 263ms	remaining: 1.11s
104:	learn: 0.0375164	total: 265ms	remaining: 1.1s
105:	learn: 0.0370523	total: 267ms	remaining: 1.1s
106:	learn: 0.0368438	total: 271ms	remaining: 1.1s
107:	learn: 0.0365629	total: 274ms	remaining: 1.1s
108:	learn: 0.0361677	total: 277ms	remaining: 1.1s
109:	learn: 0.0360146	total: 280ms	remaining: 1.1s
110:	learn: 0.0357859	total: 285ms	remaining: 1.1s
111:	learn: 0.0354980	total: 293ms	remaining: 1.13s
112:	learn: 0.0353129	total: 296ms	remaining: 1.12s
113:	learn: 0.0350455	total: 323ms	remaining: 1.21s
114:	learn: 0.0349788	total: 325ms	remaining: 1.21s
115:	learn: 0.0348924	total: 327ms	remaining: 1.2s
116:	learn: 0.0347056	total: 329ms	remaining: 1.19s
117:	learn: 0.0344778	total: 331ms	remaining: 1.19s
118:	learn: 0.0342891	total: 332ms	remaining: 1.18s
119:	learn: 0.0341195	total: 334ms	remaining: 1.18s
120:	learn: 0.0339386	total: 336ms	remaining: 1.17s
121:	learn: 0.0337933	total: 338ms	remaining: 1.16s
122:	learn: 0.0332538	total: 340ms	remaining: 1.16s
123:	learn: 0.0326110	total: 342ms	remaining: 1.15s
124:	learn: 0.0322800	total: 344ms	remaining: 1.15s
125:	learn: 0.0316953	total: 346ms	remaining: 1.14s
126:	learn: 0.0314186	total: 347ms	remaining: 1.13s
127:	learn: 0.0310274	total: 349ms	remaining: 1.13s
128:	learn: 0.0309184	total: 351ms	remaining: 1.12s
129:	learn: 0.0307077	total: 352ms	remaining: 1.12s
130:	learn: 0.0303400	total: 354ms	remaining: 1.11s
131:	learn: 0.0302933	total: 356ms	remaining: 1.11s
132:	learn: 0.0300932	total: 359ms	remaining: 1.1s
133:	learn: 0.0299046	total: 361ms	remaining: 1.1s
134:	learn: 0.0297765	total: 363ms	remaining: 1.09s
135:	learn: 0.0294156	total: 366ms	remaining: 1.09s
136:	learn: 0.0292020	total: 368ms	remaining: 1.09s
137:	learn: 0.0290537	total: 370ms	remaining: 1.08s
138:	learn: 0.0288690	total: 372ms	remaining: 1.08s
139:	learn: 0.0287612	total: 374ms	remaining: 1.07s
140:	learn: 0.0285253	total: 376ms	remaining: 1.07s
141:	learn: 0.0282760	total: 378ms	remaining: 1.06s
142:	learn: 0.0278096	total: 379ms	remaining: 1.06s
143:	learn: 0.0274291	total: 381ms	remaining: 1.05s
144:	learn: 0.0273679	total: 383ms	remaining: 1.05s
145:	learn: 0.0271021	total: 385ms	remaining: 1.04s
146:	learn: 0.0269212	total: 387ms	remaining: 1.04s
147:	learn: 0.0269004	total: 390ms	remaining: 1.04s
148:	learn: 0.0268318	total: 392ms	remaining: 1.03s
149:	learn: 0.0267039	total: 394ms	remaining: 1.03s
150:	learn: 0.0264488	total: 397ms	remaining: 1.03s
151:	learn: 0.0262861	total: 399ms	remaining: 1.02s
152:	learn: 0.0262451	total: 401ms	remaining: 1.02s
153:	learn: 0.0258393	total: 403ms	remaining: 1.01s
154:	learn: 0.0258071	total: 404ms	remaining: 1.01s
155:	learn: 0.0257759	total: 406ms	remaining: 1s
156:	learn: 0.0252997	total: 408ms	remaining: 1s
157:	learn: 0.0251353	total: 410ms	remaining: 998ms
158:	learn: 0.0249112	total: 413ms	remaining: 994ms
159:	learn: 0.0248052	total: 415ms	remaining: 991ms
160:	learn: 0.0246061	total: 417ms	remaining: 988ms
161:	learn: 0.0244320	total: 419ms	remaining: 984ms
162:	learn: 0.0242637	total: 421ms	remaining: 980ms
163:	learn: 0.0240896	total: 423ms	remaining: 975ms
164:	learn: 0.0240073	total: 425ms	remaining: 971ms
165:	learn: 0.0237043	total: 427ms	remaining: 966ms
166:	learn: 0.0234466	total: 428ms	remaining: 962ms
167:	learn: 0.0233448	total: 430ms	remaining: 958ms
168:	learn: 0.0232565	total: 432ms	remaining: 954ms
169:	learn: 0.0232087	total: 434ms	remaining: 950ms
170:	learn: 0.0229560	total: 436ms	remaining: 946ms
171:	learn: 0.0227764	total: 438ms	remaining: 942ms
172:	learn: 0.0225917	total: 440ms	remaining: 939ms
173:	learn: 0.0223844	total: 442ms	remaining: 935ms
174:	learn: 0.0223412	total: 445ms	remaining: 932ms
175:	learn: 0.0221555	total: 447ms	remaining: 930ms
176:	learn: 0.0221331	total: 449ms	remaining: 926ms
177:	learn: 0.0219202	total: 452ms	remaining: 924ms
178:	learn: 0.0215429	total: 454ms	remaining: 920ms
179:	learn: 0.0215060	total: 456ms	remaining: 918ms
180:	learn: 0.0213688	total: 458ms	remaining: 914ms
181:	learn: 0.0211688	total: 460ms	remaining: 910ms
182:	learn: 0.0210164	total: 462ms	remaining: 906ms
183:	learn: 0.0208841	total: 463ms	remaining: 902ms
184:	learn: 0.0207634	total: 465ms	remaining: 897ms
185:	learn: 0.0205252	total: 467ms	remaining: 893ms
186:	learn: 0.0202581	total: 468ms	remaining: 889ms
187:	learn: 0.0200209	total: 470ms	remaining: 886ms
188:	learn: 0.0198013	total: 472ms	remaining: 882ms
189:	learn: 0.0195307	total: 474ms	remaining: 878ms
190:	learn: 0.0195093	total: 476ms	remaining: 875ms
191:	learn: 0.0193928	total: 478ms	remaining: 871ms
192:	learn: 0.0192132	total: 479ms	remaining: 867ms
193:	learn: 0.0190335	total: 481ms	remaining: 863ms
194:	learn: 0.0189846	total: 483ms	remaining: 859ms
195:	learn: 0.0188626	total: 484ms	remaining: 855ms
196:	learn: 0.0187501	total: 486ms	remaining: 852ms
197:	learn: 0.0187345	total: 488ms	remaining: 848ms
198:	learn: 0.0185768	total: 490ms	remaining: 844ms
199:	learn: 0.0185381	total: 492ms	remaining: 841ms
200:	learn: 0.0183466	total: 493ms	remaining: 837ms
201:	learn: 0.0182416	total: 495ms	remaining: 833ms
202:	learn: 0.0181586	total: 497ms	remaining: 830ms
203:	learn: 0.0181127	total: 499ms	remaining: 826ms
204:	learn: 0.0179161	total: 500ms	remaining: 823ms
205:	learn: 0.0177340	total: 513ms	remaining: 837ms
206:	learn: 0.0175154	total: 515ms	remaining: 833ms
207:	learn: 0.0174207	total: 517ms	remaining: 830ms
208:	learn: 0.0172918	total: 519ms	remaining: 826ms
209:	learn: 0.0170658	total: 520ms	remaining: 823ms
210:	learn: 0.0170104	total: 522ms	remaining: 819ms
211:	learn: 0.0168230	total: 524ms	remaining: 815ms
212:	learn: 0.0166865	total: 526ms	remaining: 812ms
213:	learn: 0.0165908	total: 527ms	remaining: 808ms
214:	learn: 0.0163602	total: 529ms	remaining: 805ms
215:	learn: 0.0163484	total: 531ms	remaining: 801ms
216:	learn: 0.0163141	total: 532ms	remaining: 797ms
217:	learn: 0.0162660	total: 535ms	remaining: 794ms
218:	learn: 0.0161455	total: 536ms	remaining: 791ms
219:	learn: 0.0160963	total: 538ms	remaining: 788ms
220:	learn: 0.0159259	total: 540ms	remaining: 785ms
221:	learn: 0.0157545	total: 542ms	remaining: 782ms
222:	learn: 0.0156569	total: 544ms	remaining: 778ms
223:	learn: 0.0155447	total: 546ms	remaining: 775ms
224:	learn: 0.0153317	total: 548ms	remaining: 772ms
225:	learn: 0.0151548	total: 550ms	remaining: 769ms
226:	learn: 0.0150323	total: 552ms	remaining: 766ms
227:	learn: 0.0149926	total: 554ms	remaining: 763ms
228:	learn: 0.0148865	total: 556ms	remaining: 759ms
229:	learn: 0.0148735	total: 557ms	remaining: 756ms
230:	learn: 0.0147872	total: 559ms	remaining: 753ms
231:	learn: 0.0146567	total: 561ms	remaining: 749ms
232:	learn: 0.0145440	total: 563ms	remaining: 746ms
233:	learn: 0.0144196	total: 564ms	remaining: 743ms
234:	learn: 0.0142875	total: 566ms	remaining: 740ms
235:	learn: 0.0141890	total: 568ms	remaining: 736ms
236:	learn: 0.0141171	total: 570ms	remaining: 734ms
237:	learn: 0.0140023	total: 572ms	remaining: 730ms
238:	learn: 0.0139154	total: 574ms	remaining: 727ms
239:	learn: 0.0138787	total: 575ms	remaining: 724ms
240:	learn: 0.0137982	total: 577ms	remaining: 721ms
241:	learn: 0.0136988	total: 579ms	remaining: 718ms
242:	learn: 0.0135876	total: 581ms	remaining: 715ms
243:	learn: 0.0135158	total: 583ms	remaining: 712ms
244:	learn: 0.0134392	total: 585ms	remaining: 709ms
245:	learn: 0.0134018	total: 586ms	remaining: 706ms
246:	learn: 0.0133252	total: 588ms	remaining: 703ms
247:	learn: 0.0132315	total: 590ms	remaining: 700ms
248:	learn: 0.0131982	total: 592ms	remaining: 697ms
249:	learn: 0.0131700	total: 594ms	remaining: 693ms
250:	learn: 0.0131463	total: 596ms	remaining: 691ms
251:	learn: 0.0130503	total: 598ms	remaining: 689ms
252:	learn: 0.0130239	total: 601ms	remaining: 687ms
253:	learn: 0.0129336	total: 603ms	remaining: 684ms
254:	learn: 0.0129214	total: 605ms	remaining: 681ms
255:	learn: 0.0128852	total: 613ms	remaining: 685ms
256:	learn: 0.0128178	total: 615ms	remaining: 682ms
257:	learn: 0.0127754	total: 617ms	remaining: 679ms
258:	learn: 0.0127173	total: 619ms	remaining: 676ms
259:	learn: 0.0126548	total: 628ms	remaining: 681ms
260:	learn: 0.0125900	total: 630ms	remaining: 678ms
261:	learn: 0.0125346	total: 632ms	remaining: 675ms
262:	learn: 0.0124447	total: 634ms	remaining: 672ms
263:	learn: 0.0124289	total: 636ms	remaining: 669ms
264:	learn: 0.0123962	total: 674ms	remaining: 704ms
265:	learn: 0.0123819	total: 676ms	remaining: 701ms
266:	learn: 0.0123633	total: 677ms	remaining: 698ms
267:	learn: 0.0123048	total: 679ms	remaining: 694ms
268:	learn: 0.0122642	total: 681ms	remaining: 691ms
269:	learn: 0.0121602	total: 683ms	remaining: 688ms
270:	learn: 0.0120638	total: 684ms	remaining: 684ms
271:	learn: 0.0120478	total: 686ms	remaining: 681ms
272:	learn: 0.0119599	total: 688ms	remaining: 678ms
273:	learn: 0.0119360	total: 690ms	remaining: 675ms
274:	learn: 0.0119036	total: 691ms	remaining: 671ms
275:	learn: 0.0118529	total: 693ms	remaining: 668ms
276:	learn: 0.0117815	total: 695ms	remaining: 665ms
277:	learn: 0.0117299	total: 697ms	remaining: 662ms
278:	learn: 0.0116843	total: 698ms	remaining: 658ms
279:	learn: 0.0116749	total: 700ms	remaining: 655ms
280:	learn: 0.0115991	total: 702ms	remaining: 652ms
281:	learn: 0.0115518	total: 704ms	remaining: 649ms
282:	learn: 0.0115260	total: 706ms	remaining: 646ms
283:	learn: 0.0115076	total: 708ms	remaining: 643ms
284:	learn: 0.0114642	total: 710ms	remaining: 640ms
285:	learn: 0.0114211	total: 712ms	remaining: 637ms
286:	learn: 0.0113576	total: 714ms	remaining: 634ms
287:	learn: 0.0113442	total: 716ms	remaining: 631ms
288:	learn: 0.0113265	total: 717ms	remaining: 628ms
289:	learn: 0.0112635	total: 719ms	remaining: 625ms
290:	learn: 0.0111725	total: 721ms	remaining: 622ms
291:	learn: 0.0111329	total: 723ms	remaining: 619ms
292:	learn: 0.0110861	total: 725ms	remaining: 616ms
293:	learn: 0.0110681	total: 727ms	remaining: 613ms
294:	learn: 0.0110263	total: 729ms	remaining: 610ms
295:	learn: 0.0109638	total: 730ms	remaining: 607ms
296:	learn: 0.0109478	total: 732ms	remaining: 604ms
297:	learn: 0.0109127	total: 734ms	remaining: 601ms
298:	learn: 0.0108978	total: 737ms	remaining: 599ms
299:	learn: 0.0108606	total: 739ms	remaining: 596ms
300:	learn: 0.0108350	total: 746ms	remaining: 597ms
301:	learn: 0.0108005	total: 748ms	remaining: 595ms
302:	learn: 0.0108005	total: 750ms	remaining: 591ms
303:	learn: 0.0107895	total: 752ms	remaining: 589ms
304:	learn: 0.0107278	total: 753ms	remaining: 585ms
305:	learn: 0.0106400	total: 755ms	remaining: 582ms
306:	learn: 0.0106279	total: 757ms	remaining: 579ms
307:	learn: 0.0106074	total: 759ms	remaining: 576ms
308:	learn: 0.0105961	total: 760ms	remaining: 573ms
309:	learn: 0.0105860	total: 763ms	remaining: 571ms
310:	learn: 0.0105608	total: 764ms	remaining: 568ms
311:	learn: 0.0105379	total: 766ms	remaining: 565ms
312:	learn: 0.0104754	total: 768ms	remaining: 562ms
313:	learn: 0.0104470	total: 770ms	remaining: 559ms
314:	learn: 0.0103907	total: 772ms	remaining: 557ms
315:	learn: 0.0103657	total: 774ms	remaining: 554ms
316:	learn: 0.0103131	total: 776ms	remaining: 551ms
317:	learn: 0.0102567	total: 778ms	remaining: 548ms
318:	learn: 0.0101967	total: 780ms	remaining: 545ms
319:	learn: 0.0101676	total: 782ms	remaining: 542ms
320:	learn: 0.0100644	total: 783ms	remaining: 539ms
321:	learn: 0.0100295	total: 785ms	remaining: 536ms
322:	learn: 0.0100155	total: 787ms	remaining: 533ms
323:	learn: 0.0099994	total: 789ms	remaining: 531ms
324:	learn: 0.0099361	total: 791ms	remaining: 528ms
325:	learn: 0.0098944	total: 793ms	remaining: 525ms
326:	learn: 0.0098314	total: 795ms	remaining: 523ms
327:	learn: 0.0098086	total: 797ms	remaining: 520ms
328:	learn: 0.0097571	total: 799ms	remaining: 517ms
329:	learn: 0.0096993	total: 801ms	remaining: 514ms
330:	learn: 0.0096462	total: 803ms	remaining: 512ms
331:	learn: 0.0096293	total: 805ms	remaining: 509ms
332:	learn: 0.0096199	total: 807ms	remaining: 506ms
333:	learn: 0.0096126	total: 808ms	remaining: 503ms
334:	learn: 0.0095870	total: 810ms	remaining: 501ms
335:	learn: 0.0095342	total: 812ms	remaining: 498ms
336:	learn: 0.0094879	total: 814ms	remaining: 495ms
337:	learn: 0.0094417	total: 815ms	remaining: 492ms
338:	learn: 0.0093841	total: 817ms	remaining: 489ms
339:	learn: 0.0093624	total: 819ms	remaining: 487ms
340:	learn: 0.0093346	total: 821ms	remaining: 484ms
341:	learn: 0.0093149	total: 823ms	remaining: 481ms
342:	learn: 0.0093022	total: 825ms	remaining: 479ms
343:	learn: 0.0092545	total: 827ms	remaining: 476ms
344:	learn: 0.0092358	total: 829ms	remaining: 473ms
345:	learn: 0.0091875	total: 831ms	remaining: 471ms
346:	learn: 0.0091795	total: 833ms	remaining: 468ms
347:	learn: 0.0091684	total: 835ms	remaining: 465ms
348:	learn: 0.0091620	total: 837ms	remaining: 463ms
349:	learn: 0.0091440	total: 838ms	remaining: 460ms
350:	learn: 0.0091006	total: 840ms	remaining: 457ms
351:	learn: 0.0090505	total: 842ms	remaining: 454ms
352:	learn: 0.0090324	total: 843ms	remaining: 452ms
353:	learn: 0.0089966	total: 845ms	remaining: 449ms
354:	learn: 0.0089802	total: 847ms	remaining: 446ms
355:	learn: 0.0089702	total: 849ms	remaining: 443ms
356:	learn: 0.0089626	total: 850ms	remaining: 441ms
357:	learn: 0.0089460	total: 852ms	remaining: 438ms
358:	learn: 0.0088947	total: 854ms	remaining: 435ms
359:	learn: 0.0088795	total: 856ms	remaining: 433ms
360:	learn: 0.0088657	total: 857ms	remaining: 430ms
361:	learn: 0.0088355	total: 859ms	remaining: 427ms
362:	learn: 0.0088116	total: 861ms	remaining: 424ms
363:	learn: 0.0087648	total: 862ms	remaining: 422ms
364:	learn: 0.0087238	total: 864ms	remaining: 419ms
365:	learn: 0.0086798	total: 866ms	remaining: 416ms
366:	learn: 0.0086739	total: 868ms	remaining: 414ms
367:	learn: 0.0086515	total: 869ms	remaining: 411ms
368:	learn: 0.0086161	total: 871ms	remaining: 408ms
369:	learn: 0.0085699	total: 873ms	remaining: 406ms
370:	learn: 0.0085419	total: 874ms	remaining: 403ms
371:	learn: 0.0085028	total: 876ms	remaining: 400ms
372:	learn: 0.0084900	total: 878ms	remaining: 398ms
373:	learn: 0.0084585	total: 880ms	remaining: 395ms
374:	learn: 0.0084302	total: 881ms	remaining: 393ms
375:	learn: 0.0084003	total: 884ms	remaining: 390ms
376:	learn: 0.0083702	total: 886ms	remaining: 388ms
377:	learn: 0.0083270	total: 888ms	remaining: 385ms
378:	learn: 0.0083086	total: 890ms	remaining: 383ms
379:	learn: 0.0082989	total: 892ms	remaining: 380ms
380:	learn: 0.0082775	total: 894ms	remaining: 378ms
381:	learn: 0.0082690	total: 896ms	remaining: 375ms
382:	learn: 0.0082294	total: 898ms	remaining: 373ms
383:	learn: 0.0082022	total: 900ms	remaining: 370ms
384:	learn: 0.0081732	total: 902ms	remaining: 368ms
385:	learn: 0.0081625	total: 904ms	remaining: 365ms
386:	learn: 0.0081388	total: 905ms	remaining: 363ms
387:	learn: 0.0081205	total: 907ms	remaining: 360ms
388:	learn: 0.0081204	total: 908ms	remaining: 357ms
389:	learn: 0.0081106	total: 910ms	remaining: 355ms
390:	learn: 0.0080758	total: 912ms	remaining: 352ms
391:	learn: 0.0080306	total: 914ms	remaining: 350ms
392:	learn: 0.0080198	total: 916ms	remaining: 347ms
393:	learn: 0.0080141	total: 918ms	remaining: 345ms
394:	learn: 0.0079765	total: 920ms	remaining: 342ms
395:	learn: 0.0079467	total: 922ms	remaining: 340ms
396:	learn: 0.0079465	total: 923ms	remaining: 337ms
397:	learn: 0.0079374	total: 925ms	remaining: 335ms
398:	learn: 0.0079286	total: 927ms	remaining: 332ms
399:	learn: 0.0079027	total: 929ms	remaining: 330ms
400:	learn: 0.0078782	total: 931ms	remaining: 327ms
401:	learn: 0.0078435	total: 933ms	remaining: 325ms
402:	learn: 0.0078341	total: 935ms	remaining: 322ms
403:	learn: 0.0078093	total: 936ms	remaining: 320ms
404:	learn: 0.0078093	total: 938ms	remaining: 317ms
405:	learn: 0.0077741	total: 940ms	remaining: 315ms
406:	learn: 0.0077741	total: 941ms	remaining: 312ms
407:	learn: 0.0077660	total: 943ms	remaining: 310ms
408:	learn: 0.0077660	total: 944ms	remaining: 307ms
409:	learn: 0.0077582	total: 946ms	remaining: 305ms
410:	learn: 0.0077419	total: 948ms	remaining: 302ms
411:	learn: 0.0077194	total: 950ms	remaining: 300ms
412:	learn: 0.0076536	total: 951ms	remaining: 297ms
413:	learn: 0.0076210	total: 953ms	remaining: 295ms
414:	learn: 0.0076190	total: 955ms	remaining: 292ms
415:	learn: 0.0076138	total: 956ms	remaining: 290ms
416:	learn: 0.0076117	total: 958ms	remaining: 287ms
417:	learn: 0.0075811	total: 960ms	remaining: 285ms
418:	learn: 0.0075570	total: 962ms	remaining: 282ms
419:	learn: 0.0075325	total: 963ms	remaining: 280ms
420:	learn: 0.0075034	total: 965ms	remaining: 277ms
421:	learn: 0.0075015	total: 967ms	remaining: 275ms
422:	learn: 0.0074999	total: 969ms	remaining: 272ms
423:	learn: 0.0074735	total: 970ms	remaining: 270ms
424:	learn: 0.0074735	total: 972ms	remaining: 268ms
425:	learn: 0.0074507	total: 974ms	remaining: 265ms
426:	learn: 0.0074278	total: 976ms	remaining: 263ms
427:	learn: 0.0074277	total: 978ms	remaining: 260ms
428:	learn: 0.0074204	total: 979ms	remaining: 258ms
429:	learn: 0.0073817	total: 981ms	remaining: 256ms
430:	learn: 0.0073688	total: 983ms	remaining: 253ms
431:	learn: 0.0073475	total: 985ms	remaining: 251ms
432:	learn: 0.0073326	total: 987ms	remaining: 248ms
433:	learn: 0.0073306	total: 988ms	remaining: 246ms
434:	learn: 0.0073300	total: 990ms	remaining: 243ms
435:	learn: 0.0072994	total: 991ms	remaining: 241ms
436:	learn: 0.0072698	total: 993ms	remaining: 239ms
437:	learn: 0.0072697	total: 995ms	remaining: 236ms
438:	learn: 0.0072303	total: 996ms	remaining: 234ms
439:	learn: 0.0072101	total: 998ms	remaining: 231ms
440:	learn: 0.0071839	total: 1000ms	remaining: 229ms
441:	learn: 0.0071714	total: 1s	remaining: 227ms
442:	learn: 0.0071515	total: 1s	remaining: 225ms
443:	learn: 0.0071303	total: 1.01s	remaining: 222ms
444:	learn: 0.0071291	total: 1.01s	remaining: 220ms
445:	learn: 0.0071213	total: 1.01s	remaining: 218ms
446:	learn: 0.0070873	total: 1.01s	remaining: 215ms
447:	learn: 0.0070672	total: 1.02s	remaining: 214ms
448:	learn: 0.0070657	total: 1.02s	remaining: 212ms
449:	learn: 0.0070378	total: 1.02s	remaining: 209ms
450:	learn: 0.0070377	total: 1.02s	remaining: 207ms
451:	learn: 0.0070191	total: 1.03s	remaining: 204ms
452:	learn: 0.0070020	total: 1.03s	remaining: 202ms
453:	learn: 0.0070019	total: 1.03s	remaining: 200ms
454:	learn: 0.0069635	total: 1.03s	remaining: 197ms
455:	learn: 0.0069627	total: 1.03s	remaining: 195ms
456:	learn: 0.0069506	total: 1.03s	remaining: 192ms
457:	learn: 0.0069332	total: 1.04s	remaining: 190ms
458:	learn: 0.0069330	total: 1.04s	remaining: 188ms
459:	learn: 0.0069242	total: 1.04s	remaining: 185ms
460:	learn: 0.0068769	total: 1.04s	remaining: 183ms
461:	learn: 0.0068555	total: 1.04s	remaining: 181ms
462:	learn: 0.0068494	total: 1.04s	remaining: 178ms
463:	learn: 0.0068485	total: 1.04s	remaining: 176ms
464:	learn: 0.0068478	total: 1.05s	remaining: 173ms
465:	learn: 0.0068301	total: 1.05s	remaining: 171ms
466:	learn: 0.0067965	total: 1.05s	remaining: 169ms
467:	learn: 0.0067964	total: 1.05s	remaining: 166ms
468:	learn: 0.0067904	total: 1.05s	remaining: 164ms
469:	learn: 0.0067904	total: 1.05s	remaining: 162ms
470:	learn: 0.0067644	total: 1.06s	remaining: 159ms
471:	learn: 0.0067417	total: 1.06s	remaining: 157ms
472:	learn: 0.0067206	total: 1.06s	remaining: 155ms
473:	learn: 0.0066994	total: 1.06s	remaining: 152ms
474:	learn: 0.0066982	total: 1.06s	remaining: 150ms
475:	learn: 0.0066858	total: 1.06s	remaining: 148ms
476:	learn: 0.0066729	total: 1.07s	remaining: 145ms
477:	learn: 0.0066673	total: 1.07s	remaining: 143ms
478:	learn: 0.0066667	total: 1.07s	remaining: 141ms
479:	learn: 0.0066611	total: 1.07s	remaining: 138ms
480:	learn: 0.0066570	total: 1.07s	remaining: 136ms
481:	learn: 0.0066569	total: 1.07s	remaining: 134ms
482:	learn: 0.0066569	total: 1.08s	remaining: 131ms
483:	learn: 0.0066569	total: 1.08s	remaining: 129ms
484:	learn: 0.0066560	total: 1.08s	remaining: 127ms
485:	learn: 0.0066244	total: 1.08s	remaining: 125ms
486:	learn: 0.0066242	total: 1.08s	remaining: 122ms
487:	learn: 0.0066232	total: 1.08s	remaining: 120ms
488:	learn: 0.0066228	total: 1.09s	remaining: 118ms
489:	learn: 0.0066227	total: 1.09s	remaining: 116ms
490:	learn: 0.0066227	total: 1.09s	remaining: 113ms
491:	learn: 0.0066180	total: 1.09s	remaining: 111ms
492:	learn: 0.0065959	total: 1.09s	remaining: 109ms
493:	learn: 0.0065959	total: 1.09s	remaining: 106ms
494:	learn: 0.0065727	total: 1.1s	remaining: 104ms
495:	learn: 0.0065493	total: 1.1s	remaining: 102ms
496:	learn: 0.0065274	total: 1.1s	remaining: 99.7ms
497:	learn: 0.0065085	total: 1.1s	remaining: 97.4ms
498:	learn: 0.0064874	total: 1.1s	remaining: 95.2ms
499:	learn: 0.0064669	total: 1.11s	remaining: 93ms
500:	learn: 0.0064377	total: 1.11s	remaining: 90.7ms
501:	learn: 0.0064377	total: 1.11s	remaining: 88.5ms
502:	learn: 0.0064150	total: 1.11s	remaining: 86.2ms
503:	learn: 0.0063955	total: 1.11s	remaining: 84ms
504:	learn: 0.0063779	total: 1.11s	remaining: 81.8ms
505:	learn: 0.0063619	total: 1.12s	remaining: 79.5ms
506:	learn: 0.0063325	total: 1.12s	remaining: 77.3ms
507:	learn: 0.0062995	total: 1.12s	remaining: 75.1ms
508:	learn: 0.0062946	total: 1.12s	remaining: 72.8ms
509:	learn: 0.0062705	total: 1.13s	remaining: 70.6ms
510:	learn: 0.0062615	total: 1.13s	remaining: 68.4ms
511:	learn: 0.0062614	total: 1.13s	remaining: 66.1ms
512:	learn: 0.0062612	total: 1.13s	remaining: 63.9ms
513:	learn: 0.0062528	total: 1.13s	remaining: 61.6ms
514:	learn: 0.0062528	total: 1.13s	remaining: 59.4ms
515:	learn: 0.0062454	total: 1.13s	remaining: 57.2ms
516:	learn: 0.0062386	total: 1.14s	remaining: 55ms
517:	learn: 0.0062379	total: 1.14s	remaining: 52.8ms
518:	learn: 0.0062254	total: 1.14s	remaining: 50.5ms
519:	learn: 0.0062150	total: 1.14s	remaining: 48.3ms
520:	learn: 0.0062150	total: 1.14s	remaining: 46.1ms
521:	learn: 0.0062149	total: 1.15s	remaining: 43.9ms
522:	learn: 0.0062148	total: 1.15s	remaining: 41.7ms
523:	learn: 0.0062147	total: 1.15s	remaining: 39.5ms
524:	learn: 0.0062146	total: 1.15s	remaining: 37.3ms
525:	learn: 0.0062145	total: 1.15s	remaining: 35.1ms
526:	learn: 0.0062135	total: 1.15s	remaining: 32.9ms
527:	learn: 0.0062073	total: 1.16s	remaining: 30.6ms
528:	learn: 0.0062072	total: 1.16s	remaining: 28.4ms
529:	learn: 0.0061955	total: 1.16s	remaining: 26.2ms
530:	learn: 0.0061954	total: 1.16s	remaining: 24ms
531:	learn: 0.0061730	total: 1.16s	remaining: 21.8ms
532:	learn: 0.0061536	total: 1.16s	remaining: 19.7ms
533:	learn: 0.0061434	total: 1.17s	remaining: 17.5ms
534:	learn: 0.0061411	total: 1.17s	remaining: 15.3ms
535:	learn: 0.0061411	total: 1.17s	remaining: 13.1ms
536:	learn: 0.0061400	total: 1.17s	remaining: 10.9ms
537:	learn: 0.0061391	total: 1.17s	remaining: 8.72ms
538:	learn: 0.0061389	total: 1.17s	remaining: 6.54ms
539:	learn: 0.0061030	total: 1.18s	remaining: 4.35ms
540:	learn: 0.0061020	total: 1.18s	remaining: 2.18ms
541:	learn: 0.0061020	total: 1.18s	remaining: 0us
0:	learn: 0.5480366	total: 2.03ms	remaining: 1.1s
1:	learn: 0.4624589	total: 4.14ms	remaining: 1.12s
2:	learn: 0.4017939	total: 6.15ms	remaining: 1.1s
3:	learn: 0.3609701	total: 7.99ms	remaining: 1.07s
4:	learn: 0.3266696	total: 9.94ms	remaining: 1.07s
5:	learn: 0.3079548	total: 11.6ms	remaining: 1.04s
6:	learn: 0.2943739	total: 13.9ms	remaining: 1.06s
7:	learn: 0.2786337	total: 15.6ms	remaining: 1.04s
8:	learn: 0.2653844	total: 17.5ms	remaining: 1.04s
9:	learn: 0.2561433	total: 19.2ms	remaining: 1.02s
10:	learn: 0.2421510	total: 20.8ms	remaining: 1s
11:	learn: 0.2356686	total: 22.3ms	remaining: 985ms
12:	learn: 0.2258600	total: 24ms	remaining: 977ms
13:	learn: 0.2193136	total: 25.8ms	remaining: 972ms
14:	learn: 0.2090478	total: 27.5ms	remaining: 965ms
15:	learn: 0.1990510	total: 29.3ms	remaining: 963ms
16:	learn: 0.1938965	total: 31.1ms	remaining: 959ms
17:	learn: 0.1876548	total: 33.1ms	remaining: 962ms
18:	learn: 0.1829584	total: 34.7ms	remaining: 956ms
19:	learn: 0.1796156	total: 36.4ms	remaining: 950ms
20:	learn: 0.1745745	total: 37.9ms	remaining: 941ms
21:	learn: 0.1720562	total: 39.5ms	remaining: 933ms
22:	learn: 0.1710136	total: 41ms	remaining: 926ms
23:	learn: 0.1666749	total: 42.6ms	remaining: 919ms
24:	learn: 0.1613101	total: 44.2ms	remaining: 913ms
25:	learn: 0.1572853	total: 45.8ms	remaining: 908ms
26:	learn: 0.1528608	total: 47.4ms	remaining: 904ms
27:	learn: 0.1493365	total: 49ms	remaining: 899ms
28:	learn: 0.1463798	total: 51ms	remaining: 902ms
29:	learn: 0.1433626	total: 52.6ms	remaining: 897ms
30:	learn: 0.1425352	total: 54.2ms	remaining: 893ms
31:	learn: 0.1385804	total: 55.8ms	remaining: 889ms
32:	learn: 0.1366617	total: 57.3ms	remaining: 884ms
33:	learn: 0.1354031	total: 59ms	remaining: 882ms
34:	learn: 0.1308443	total: 60.7ms	remaining: 879ms
35:	learn: 0.1271884	total: 62.5ms	remaining: 879ms
36:	learn: 0.1248429	total: 64.7ms	remaining: 883ms
37:	learn: 0.1244923	total: 66.7ms	remaining: 884ms
38:	learn: 0.1206140	total: 68.2ms	remaining: 880ms
39:	learn: 0.1148401	total: 69.9ms	remaining: 877ms
40:	learn: 0.1131167	total: 71.6ms	remaining: 875ms
41:	learn: 0.1119704	total: 73.1ms	remaining: 870ms
42:	learn: 0.1068329	total: 74.8ms	remaining: 868ms
43:	learn: 0.1049105	total: 76.3ms	remaining: 864ms
44:	learn: 0.1039569	total: 77.8ms	remaining: 859ms
45:	learn: 0.1027078	total: 79.6ms	remaining: 858ms
46:	learn: 0.0986905	total: 81.2ms	remaining: 855ms
47:	learn: 0.0980221	total: 83.1ms	remaining: 855ms
48:	learn: 0.0946121	total: 84.7ms	remaining: 852ms
49:	learn: 0.0918321	total: 86.2ms	remaining: 849ms
50:	learn: 0.0893413	total: 88ms	remaining: 847ms
51:	learn: 0.0870765	total: 89.5ms	remaining: 844ms
52:	learn: 0.0838528	total: 91.2ms	remaining: 841ms
53:	learn: 0.0829213	total: 93ms	remaining: 841ms
54:	learn: 0.0823812	total: 94.9ms	remaining: 840ms
55:	learn: 0.0810166	total: 96.7ms	remaining: 839ms
56:	learn: 0.0797774	total: 98.3ms	remaining: 836ms
57:	learn: 0.0789854	total: 100ms	remaining: 834ms
58:	learn: 0.0769708	total: 102ms	remaining: 833ms
59:	learn: 0.0745340	total: 104ms	remaining: 834ms
60:	learn: 0.0727207	total: 106ms	remaining: 835ms
61:	learn: 0.0714103	total: 109ms	remaining: 842ms
62:	learn: 0.0697223	total: 111ms	remaining: 847ms
63:	learn: 0.0676244	total: 114ms	remaining: 848ms
64:	learn: 0.0665914	total: 116ms	remaining: 848ms
65:	learn: 0.0660735	total: 118ms	remaining: 848ms
66:	learn: 0.0646071	total: 120ms	remaining: 850ms
67:	learn: 0.0635169	total: 122ms	remaining: 850ms
68:	learn: 0.0629419	total: 125ms	remaining: 856ms
69:	learn: 0.0614332	total: 127ms	remaining: 857ms
70:	learn: 0.0604219	total: 129ms	remaining: 856ms
71:	learn: 0.0599668	total: 131ms	remaining: 855ms
72:	learn: 0.0589767	total: 133ms	remaining: 855ms
73:	learn: 0.0586753	total: 135ms	remaining: 856ms
74:	learn: 0.0576200	total: 138ms	remaining: 857ms
75:	learn: 0.0568633	total: 140ms	remaining: 856ms
76:	learn: 0.0567398	total: 142ms	remaining: 856ms
77:	learn: 0.0562199	total: 144ms	remaining: 854ms
78:	learn: 0.0560434	total: 145ms	remaining: 851ms
79:	learn: 0.0550781	total: 147ms	remaining: 849ms
80:	learn: 0.0536519	total: 149ms	remaining: 847ms
81:	learn: 0.0529180	total: 151ms	remaining: 846ms
82:	learn: 0.0516731	total: 153ms	remaining: 844ms
83:	learn: 0.0504839	total: 155ms	remaining: 846ms
84:	learn: 0.0492326	total: 157ms	remaining: 846ms
85:	learn: 0.0480230	total: 169ms	remaining: 893ms
86:	learn: 0.0469825	total: 170ms	remaining: 892ms
87:	learn: 0.0460676	total: 172ms	remaining: 888ms
88:	learn: 0.0451183	total: 174ms	remaining: 885ms
89:	learn: 0.0442304	total: 176ms	remaining: 883ms
90:	learn: 0.0433947	total: 178ms	remaining: 881ms
91:	learn: 0.0428872	total: 179ms	remaining: 877ms
92:	learn: 0.0424879	total: 181ms	remaining: 876ms
93:	learn: 0.0420935	total: 184ms	remaining: 876ms
94:	learn: 0.0411913	total: 186ms	remaining: 876ms
95:	learn: 0.0402884	total: 188ms	remaining: 874ms
96:	learn: 0.0396769	total: 190ms	remaining: 873ms
97:	learn: 0.0393285	total: 192ms	remaining: 870ms
98:	learn: 0.0392216	total: 194ms	remaining: 868ms
99:	learn: 0.0386305	total: 196ms	remaining: 867ms
100:	learn: 0.0380668	total: 199ms	remaining: 867ms
101:	learn: 0.0375952	total: 201ms	remaining: 867ms
102:	learn: 0.0367326	total: 203ms	remaining: 864ms
103:	learn: 0.0359664	total: 205ms	remaining: 862ms
104:	learn: 0.0355020	total: 206ms	remaining: 859ms
105:	learn: 0.0349996	total: 208ms	remaining: 857ms
106:	learn: 0.0346189	total: 210ms	remaining: 855ms
107:	learn: 0.0338905	total: 213ms	remaining: 854ms
108:	learn: 0.0333696	total: 215ms	remaining: 853ms
109:	learn: 0.0330784	total: 217ms	remaining: 851ms
110:	learn: 0.0327062	total: 219ms	remaining: 850ms
111:	learn: 0.0324114	total: 242ms	remaining: 928ms
112:	learn: 0.0320925	total: 244ms	remaining: 925ms
113:	learn: 0.0314929	total: 245ms	remaining: 921ms
114:	learn: 0.0310075	total: 247ms	remaining: 918ms
115:	learn: 0.0305238	total: 249ms	remaining: 914ms
116:	learn: 0.0301088	total: 251ms	remaining: 910ms
117:	learn: 0.0295559	total: 252ms	remaining: 907ms
118:	learn: 0.0293485	total: 254ms	remaining: 904ms
119:	learn: 0.0290087	total: 256ms	remaining: 902ms
120:	learn: 0.0287385	total: 259ms	remaining: 901ms
121:	learn: 0.0284376	total: 262ms	remaining: 901ms
122:	learn: 0.0281099	total: 264ms	remaining: 899ms
123:	learn: 0.0278641	total: 266ms	remaining: 897ms
124:	learn: 0.0276192	total: 268ms	remaining: 895ms
125:	learn: 0.0274918	total: 271ms	remaining: 894ms
126:	learn: 0.0273646	total: 273ms	remaining: 892ms
127:	learn: 0.0271158	total: 275ms	remaining: 889ms
128:	learn: 0.0270794	total: 277ms	remaining: 887ms
129:	learn: 0.0268323	total: 279ms	remaining: 885ms
130:	learn: 0.0265601	total: 281ms	remaining: 882ms
131:	learn: 0.0262280	total: 283ms	remaining: 879ms
132:	learn: 0.0258531	total: 285ms	remaining: 878ms
133:	learn: 0.0255690	total: 288ms	remaining: 878ms
134:	learn: 0.0254396	total: 291ms	remaining: 876ms
135:	learn: 0.0251595	total: 293ms	remaining: 874ms
136:	learn: 0.0251149	total: 295ms	remaining: 873ms
137:	learn: 0.0250868	total: 297ms	remaining: 871ms
138:	learn: 0.0249868	total: 300ms	remaining: 869ms
139:	learn: 0.0249460	total: 302ms	remaining: 867ms
140:	learn: 0.0248242	total: 304ms	remaining: 864ms
141:	learn: 0.0244354	total: 306ms	remaining: 862ms
142:	learn: 0.0244120	total: 308ms	remaining: 859ms
143:	learn: 0.0241705	total: 310ms	remaining: 857ms
144:	learn: 0.0240996	total: 312ms	remaining: 854ms
145:	learn: 0.0240341	total: 328ms	remaining: 889ms
146:	learn: 0.0237021	total: 330ms	remaining: 886ms
147:	learn: 0.0236552	total: 332ms	remaining: 883ms
148:	learn: 0.0234600	total: 334ms	remaining: 880ms
149:	learn: 0.0230861	total: 335ms	remaining: 876ms
150:	learn: 0.0229565	total: 337ms	remaining: 872ms
151:	learn: 0.0229224	total: 338ms	remaining: 868ms
152:	learn: 0.0226775	total: 340ms	remaining: 864ms
153:	learn: 0.0226451	total: 342ms	remaining: 861ms
154:	learn: 0.0226141	total: 344ms	remaining: 858ms
155:	learn: 0.0224027	total: 346ms	remaining: 856ms
156:	learn: 0.0221895	total: 348ms	remaining: 853ms
157:	learn: 0.0219525	total: 350ms	remaining: 850ms
158:	learn: 0.0216195	total: 351ms	remaining: 847ms
159:	learn: 0.0213651	total: 353ms	remaining: 843ms
160:	learn: 0.0211732	total: 355ms	remaining: 840ms
161:	learn: 0.0208394	total: 356ms	remaining: 836ms
162:	learn: 0.0206557	total: 358ms	remaining: 833ms
163:	learn: 0.0204355	total: 360ms	remaining: 829ms
164:	learn: 0.0202374	total: 361ms	remaining: 826ms
165:	learn: 0.0200653	total: 363ms	remaining: 822ms
166:	learn: 0.0198004	total: 365ms	remaining: 819ms
167:	learn: 0.0196467	total: 367ms	remaining: 816ms
168:	learn: 0.0194756	total: 368ms	remaining: 812ms
169:	learn: 0.0193456	total: 370ms	remaining: 809ms
170:	learn: 0.0192214	total: 372ms	remaining: 806ms
171:	learn: 0.0190733	total: 374ms	remaining: 804ms
172:	learn: 0.0189332	total: 376ms	remaining: 801ms
173:	learn: 0.0187872	total: 378ms	remaining: 799ms
174:	learn: 0.0186611	total: 380ms	remaining: 796ms
175:	learn: 0.0186562	total: 386ms	remaining: 804ms
176:	learn: 0.0183772	total: 388ms	remaining: 800ms
177:	learn: 0.0182511	total: 390ms	remaining: 798ms
178:	learn: 0.0181497	total: 392ms	remaining: 794ms
179:	learn: 0.0180278	total: 394ms	remaining: 791ms
180:	learn: 0.0178910	total: 395ms	remaining: 788ms
181:	learn: 0.0177909	total: 397ms	remaining: 785ms
182:	learn: 0.0177627	total: 398ms	remaining: 782ms
183:	learn: 0.0176413	total: 401ms	remaining: 779ms
184:	learn: 0.0175007	total: 403ms	remaining: 777ms
185:	learn: 0.0173837	total: 404ms	remaining: 774ms
186:	learn: 0.0172722	total: 406ms	remaining: 771ms
187:	learn: 0.0171816	total: 408ms	remaining: 768ms
188:	learn: 0.0170598	total: 410ms	remaining: 765ms
189:	learn: 0.0169603	total: 411ms	remaining: 762ms
190:	learn: 0.0168365	total: 413ms	remaining: 759ms
191:	learn: 0.0167412	total: 415ms	remaining: 757ms
192:	learn: 0.0166920	total: 430ms	remaining: 778ms
193:	learn: 0.0165754	total: 432ms	remaining: 775ms
194:	learn: 0.0164746	total: 434ms	remaining: 772ms
195:	learn: 0.0163835	total: 435ms	remaining: 768ms
196:	learn: 0.0163602	total: 437ms	remaining: 765ms
197:	learn: 0.0162224	total: 439ms	remaining: 762ms
198:	learn: 0.0161397	total: 441ms	remaining: 760ms
199:	learn: 0.0161282	total: 443ms	remaining: 757ms
200:	learn: 0.0160415	total: 444ms	remaining: 754ms
201:	learn: 0.0159438	total: 446ms	remaining: 751ms
202:	learn: 0.0159089	total: 448ms	remaining: 748ms
203:	learn: 0.0158201	total: 450ms	remaining: 745ms
204:	learn: 0.0157394	total: 451ms	remaining: 741ms
205:	learn: 0.0157191	total: 453ms	remaining: 738ms
206:	learn: 0.0156038	total: 454ms	remaining: 735ms
207:	learn: 0.0155725	total: 456ms	remaining: 732ms
208:	learn: 0.0154310	total: 457ms	remaining: 728ms
209:	learn: 0.0153290	total: 459ms	remaining: 725ms
210:	learn: 0.0152401	total: 461ms	remaining: 723ms
211:	learn: 0.0151995	total: 462ms	remaining: 720ms
212:	learn: 0.0151995	total: 464ms	remaining: 716ms
213:	learn: 0.0151165	total: 466ms	remaining: 714ms
214:	learn: 0.0150260	total: 467ms	remaining: 711ms
215:	learn: 0.0148858	total: 469ms	remaining: 708ms
216:	learn: 0.0148655	total: 471ms	remaining: 705ms
217:	learn: 0.0148093	total: 473ms	remaining: 703ms
218:	learn: 0.0147387	total: 475ms	remaining: 701ms
219:	learn: 0.0147089	total: 477ms	remaining: 698ms
220:	learn: 0.0147063	total: 479ms	remaining: 695ms
221:	learn: 0.0146217	total: 481ms	remaining: 693ms
222:	learn: 0.0145360	total: 483ms	remaining: 691ms
223:	learn: 0.0143419	total: 485ms	remaining: 688ms
224:	learn: 0.0142695	total: 487ms	remaining: 686ms
225:	learn: 0.0141952	total: 488ms	remaining: 683ms
226:	learn: 0.0140837	total: 490ms	remaining: 680ms
227:	learn: 0.0139740	total: 492ms	remaining: 677ms
228:	learn: 0.0139538	total: 494ms	remaining: 675ms
229:	learn: 0.0138855	total: 495ms	remaining: 672ms
230:	learn: 0.0137558	total: 497ms	remaining: 669ms
231:	learn: 0.0137283	total: 499ms	remaining: 667ms
232:	learn: 0.0137279	total: 501ms	remaining: 665ms
233:	learn: 0.0137026	total: 503ms	remaining: 662ms
234:	learn: 0.0135861	total: 505ms	remaining: 660ms
235:	learn: 0.0134721	total: 507ms	remaining: 657ms
236:	learn: 0.0134464	total: 508ms	remaining: 654ms
237:	learn: 0.0134226	total: 510ms	remaining: 652ms
238:	learn: 0.0132749	total: 512ms	remaining: 649ms
239:	learn: 0.0132733	total: 514ms	remaining: 646ms
240:	learn: 0.0132721	total: 515ms	remaining: 644ms
241:	learn: 0.0132699	total: 517ms	remaining: 641ms
242:	learn: 0.0132676	total: 519ms	remaining: 638ms
243:	learn: 0.0132655	total: 521ms	remaining: 636ms
244:	learn: 0.0132405	total: 522ms	remaining: 633ms
245:	learn: 0.0131618	total: 524ms	remaining: 631ms
246:	learn: 0.0130671	total: 526ms	remaining: 628ms
247:	learn: 0.0129914	total: 527ms	remaining: 625ms
248:	learn: 0.0129699	total: 529ms	remaining: 623ms
249:	learn: 0.0129110	total: 531ms	remaining: 621ms
250:	learn: 0.0128688	total: 534ms	remaining: 619ms
251:	learn: 0.0128507	total: 536ms	remaining: 616ms
252:	learn: 0.0128472	total: 537ms	remaining: 614ms
253:	learn: 0.0127741	total: 539ms	remaining: 611ms
254:	learn: 0.0127641	total: 541ms	remaining: 609ms
255:	learn: 0.0127638	total: 542ms	remaining: 606ms
256:	learn: 0.0127119	total: 544ms	remaining: 604ms
257:	learn: 0.0126305	total: 546ms	remaining: 601ms
258:	learn: 0.0125612	total: 548ms	remaining: 599ms
259:	learn: 0.0124689	total: 550ms	remaining: 597ms
260:	learn: 0.0123931	total: 552ms	remaining: 594ms
261:	learn: 0.0123478	total: 554ms	remaining: 592ms
262:	learn: 0.0122600	total: 556ms	remaining: 590ms
263:	learn: 0.0121269	total: 558ms	remaining: 588ms
264:	learn: 0.0120470	total: 561ms	remaining: 586ms
265:	learn: 0.0119614	total: 563ms	remaining: 584ms
266:	learn: 0.0119105	total: 565ms	remaining: 582ms
267:	learn: 0.0119105	total: 567ms	remaining: 579ms
268:	learn: 0.0118054	total: 569ms	remaining: 577ms
269:	learn: 0.0117473	total: 571ms	remaining: 575ms
270:	learn: 0.0116777	total: 573ms	remaining: 573ms
271:	learn: 0.0116315	total: 575ms	remaining: 571ms
272:	learn: 0.0115604	total: 577ms	remaining: 569ms
273:	learn: 0.0115492	total: 579ms	remaining: 566ms
274:	learn: 0.0114396	total: 581ms	remaining: 564ms
275:	learn: 0.0113921	total: 583ms	remaining: 562ms
276:	learn: 0.0113483	total: 584ms	remaining: 559ms
277:	learn: 0.0112891	total: 586ms	remaining: 557ms
278:	learn: 0.0112333	total: 588ms	remaining: 554ms
279:	learn: 0.0112333	total: 590ms	remaining: 552ms
280:	learn: 0.0111565	total: 592ms	remaining: 550ms
281:	learn: 0.0110998	total: 594ms	remaining: 547ms
282:	learn: 0.0110997	total: 595ms	remaining: 545ms
283:	learn: 0.0110262	total: 597ms	remaining: 542ms
284:	learn: 0.0110106	total: 599ms	remaining: 540ms
285:	learn: 0.0109702	total: 601ms	remaining: 538ms
286:	learn: 0.0109703	total: 603ms	remaining: 535ms
287:	learn: 0.0109325	total: 605ms	remaining: 533ms
288:	learn: 0.0108753	total: 606ms	remaining: 531ms
289:	learn: 0.0107767	total: 608ms	remaining: 528ms
290:	learn: 0.0106815	total: 610ms	remaining: 526ms
291:	learn: 0.0106373	total: 612ms	remaining: 524ms
292:	learn: 0.0105756	total: 613ms	remaining: 521ms
293:	learn: 0.0105755	total: 615ms	remaining: 518ms
294:	learn: 0.0105162	total: 616ms	remaining: 516ms
295:	learn: 0.0104976	total: 618ms	remaining: 514ms
296:	learn: 0.0104663	total: 620ms	remaining: 512ms
297:	learn: 0.0104489	total: 622ms	remaining: 509ms
298:	learn: 0.0104427	total: 624ms	remaining: 507ms
299:	learn: 0.0103748	total: 625ms	remaining: 505ms
300:	learn: 0.0103210	total: 627ms	remaining: 502ms
301:	learn: 0.0103209	total: 629ms	remaining: 500ms
302:	learn: 0.0102857	total: 631ms	remaining: 497ms
303:	learn: 0.0102230	total: 632ms	remaining: 495ms
304:	learn: 0.0101655	total: 634ms	remaining: 493ms
305:	learn: 0.0101155	total: 637ms	remaining: 491ms
306:	learn: 0.0101155	total: 639ms	remaining: 489ms
307:	learn: 0.0101154	total: 641ms	remaining: 487ms
308:	learn: 0.0101147	total: 643ms	remaining: 485ms
309:	learn: 0.0101138	total: 645ms	remaining: 482ms
310:	learn: 0.0101137	total: 646ms	remaining: 480ms
311:	learn: 0.0101135	total: 648ms	remaining: 478ms
312:	learn: 0.0101135	total: 649ms	remaining: 475ms
313:	learn: 0.0101135	total: 651ms	remaining: 472ms
314:	learn: 0.0101135	total: 652ms	remaining: 470ms
315:	learn: 0.0101132	total: 654ms	remaining: 468ms
316:	learn: 0.0101129	total: 655ms	remaining: 465ms
317:	learn: 0.0100544	total: 657ms	remaining: 463ms
318:	learn: 0.0100478	total: 659ms	remaining: 460ms
319:	learn: 0.0099500	total: 661ms	remaining: 458ms
320:	learn: 0.0098863	total: 663ms	remaining: 456ms
321:	learn: 0.0098767	total: 665ms	remaining: 454ms
322:	learn: 0.0098766	total: 667ms	remaining: 452ms
323:	learn: 0.0098329	total: 670ms	remaining: 451ms
324:	learn: 0.0097969	total: 672ms	remaining: 448ms
325:	learn: 0.0097970	total: 674ms	remaining: 446ms
326:	learn: 0.0097561	total: 676ms	remaining: 444ms
327:	learn: 0.0097013	total: 678ms	remaining: 442ms
328:	learn: 0.0096637	total: 680ms	remaining: 440ms
329:	learn: 0.0095992	total: 681ms	remaining: 438ms
330:	learn: 0.0095516	total: 683ms	remaining: 436ms
331:	learn: 0.0095198	total: 685ms	remaining: 433ms
332:	learn: 0.0094736	total: 687ms	remaining: 431ms
333:	learn: 0.0094453	total: 689ms	remaining: 429ms
334:	learn: 0.0094129	total: 691ms	remaining: 427ms
335:	learn: 0.0093615	total: 693ms	remaining: 425ms
336:	learn: 0.0093141	total: 695ms	remaining: 423ms
337:	learn: 0.0092827	total: 697ms	remaining: 421ms
338:	learn: 0.0092410	total: 699ms	remaining: 418ms
339:	learn: 0.0092410	total: 700ms	remaining: 416ms
340:	learn: 0.0092099	total: 703ms	remaining: 414ms
341:	learn: 0.0091805	total: 705ms	remaining: 412ms
342:	learn: 0.0091495	total: 707ms	remaining: 410ms
343:	learn: 0.0091037	total: 709ms	remaining: 408ms
344:	learn: 0.0090757	total: 710ms	remaining: 406ms
345:	learn: 0.0090448	total: 712ms	remaining: 403ms
346:	learn: 0.0089725	total: 713ms	remaining: 401ms
347:	learn: 0.0089726	total: 715ms	remaining: 398ms
348:	learn: 0.0088980	total: 717ms	remaining: 396ms
349:	learn: 0.0088676	total: 719ms	remaining: 394ms
350:	learn: 0.0088262	total: 720ms	remaining: 392ms
351:	learn: 0.0088262	total: 722ms	remaining: 390ms
352:	learn: 0.0088100	total: 724ms	remaining: 388ms
353:	learn: 0.0087728	total: 726ms	remaining: 386ms
354:	learn: 0.0087728	total: 728ms	remaining: 383ms
355:	learn: 0.0087266	total: 729ms	remaining: 381ms
356:	learn: 0.0087213	total: 731ms	remaining: 379ms
357:	learn: 0.0087209	total: 733ms	remaining: 377ms
358:	learn: 0.0086954	total: 735ms	remaining: 375ms
359:	learn: 0.0086643	total: 737ms	remaining: 372ms
360:	learn: 0.0086296	total: 739ms	remaining: 370ms
361:	learn: 0.0086296	total: 740ms	remaining: 368ms
362:	learn: 0.0086010	total: 742ms	remaining: 366ms
363:	learn: 0.0085593	total: 744ms	remaining: 364ms
364:	learn: 0.0085323	total: 745ms	remaining: 361ms
365:	learn: 0.0084940	total: 748ms	remaining: 360ms
366:	learn: 0.0084579	total: 750ms	remaining: 358ms
367:	learn: 0.0084338	total: 752ms	remaining: 355ms
368:	learn: 0.0083976	total: 754ms	remaining: 353ms
369:	learn: 0.0083525	total: 756ms	remaining: 352ms
370:	learn: 0.0083525	total: 758ms	remaining: 349ms
371:	learn: 0.0083229	total: 760ms	remaining: 347ms
372:	learn: 0.0083228	total: 762ms	remaining: 345ms
373:	learn: 0.0082827	total: 764ms	remaining: 343ms
374:	learn: 0.0082565	total: 767ms	remaining: 341ms
375:	learn: 0.0082211	total: 769ms	remaining: 339ms
376:	learn: 0.0082068	total: 771ms	remaining: 337ms
377:	learn: 0.0081862	total: 772ms	remaining: 335ms
378:	learn: 0.0081861	total: 774ms	remaining: 333ms
379:	learn: 0.0081581	total: 776ms	remaining: 331ms
380:	learn: 0.0081564	total: 778ms	remaining: 329ms
381:	learn: 0.0081312	total: 780ms	remaining: 326ms
382:	learn: 0.0081290	total: 781ms	remaining: 324ms
383:	learn: 0.0081290	total: 783ms	remaining: 322ms
384:	learn: 0.0081290	total: 785ms	remaining: 320ms
385:	learn: 0.0081289	total: 786ms	remaining: 318ms
386:	learn: 0.0081290	total: 788ms	remaining: 316ms
387:	learn: 0.0081289	total: 789ms	remaining: 313ms
388:	learn: 0.0081289	total: 791ms	remaining: 311ms
389:	learn: 0.0081289	total: 792ms	remaining: 309ms
390:	learn: 0.0081286	total: 794ms	remaining: 307ms
391:	learn: 0.0081285	total: 795ms	remaining: 304ms
392:	learn: 0.0081284	total: 797ms	remaining: 302ms
393:	learn: 0.0081283	total: 799ms	remaining: 300ms
394:	learn: 0.0081283	total: 800ms	remaining: 298ms
395:	learn: 0.0081281	total: 802ms	remaining: 296ms
396:	learn: 0.0081280	total: 804ms	remaining: 294ms
397:	learn: 0.0081278	total: 806ms	remaining: 292ms
398:	learn: 0.0081278	total: 807ms	remaining: 289ms
399:	learn: 0.0081278	total: 809ms	remaining: 287ms
400:	learn: 0.0081275	total: 811ms	remaining: 285ms
401:	learn: 0.0081275	total: 813ms	remaining: 283ms
402:	learn: 0.0081272	total: 815ms	remaining: 281ms
403:	learn: 0.0081255	total: 818ms	remaining: 279ms
404:	learn: 0.0081232	total: 820ms	remaining: 277ms
405:	learn: 0.0081205	total: 822ms	remaining: 275ms
406:	learn: 0.0081181	total: 825ms	remaining: 274ms
407:	learn: 0.0081180	total: 826ms	remaining: 271ms
408:	learn: 0.0081156	total: 828ms	remaining: 269ms
409:	learn: 0.0081154	total: 830ms	remaining: 267ms
410:	learn: 0.0081140	total: 832ms	remaining: 265ms
411:	learn: 0.0080879	total: 834ms	remaining: 263ms
412:	learn: 0.0080879	total: 835ms	remaining: 261ms
413:	learn: 0.0080866	total: 837ms	remaining: 259ms
414:	learn: 0.0080840	total: 838ms	remaining: 257ms
415:	learn: 0.0080594	total: 840ms	remaining: 254ms
416:	learn: 0.0080549	total: 842ms	remaining: 252ms
417:	learn: 0.0080327	total: 844ms	remaining: 250ms
418:	learn: 0.0080325	total: 846ms	remaining: 248ms
419:	learn: 0.0080050	total: 848ms	remaining: 246ms
420:	learn: 0.0080050	total: 849ms	remaining: 244ms
421:	learn: 0.0079661	total: 851ms	remaining: 242ms
422:	learn: 0.0079644	total: 853ms	remaining: 240ms
423:	learn: 0.0079273	total: 876ms	remaining: 244ms
424:	learn: 0.0078908	total: 878ms	remaining: 242ms
425:	learn: 0.0078614	total: 880ms	remaining: 240ms
426:	learn: 0.0078614	total: 881ms	remaining: 237ms
427:	learn: 0.0078392	total: 883ms	remaining: 235ms
428:	learn: 0.0078042	total: 885ms	remaining: 233ms
429:	learn: 0.0077721	total: 886ms	remaining: 231ms
430:	learn: 0.0077306	total: 888ms	remaining: 229ms
431:	learn: 0.0077062	total: 890ms	remaining: 227ms
432:	learn: 0.0077062	total: 891ms	remaining: 224ms
433:	learn: 0.0076842	total: 893ms	remaining: 222ms
434:	learn: 0.0076610	total: 895ms	remaining: 220ms
435:	learn: 0.0076373	total: 896ms	remaining: 218ms
436:	learn: 0.0076108	total: 898ms	remaining: 216ms
437:	learn: 0.0075863	total: 900ms	remaining: 214ms
438:	learn: 0.0075863	total: 901ms	remaining: 211ms
439:	learn: 0.0075590	total: 903ms	remaining: 209ms
440:	learn: 0.0075553	total: 906ms	remaining: 207ms
441:	learn: 0.0075256	total: 908ms	remaining: 205ms
442:	learn: 0.0075041	total: 910ms	remaining: 203ms
443:	learn: 0.0074742	total: 912ms	remaining: 201ms
444:	learn: 0.0074497	total: 914ms	remaining: 199ms
445:	learn: 0.0074321	total: 918ms	remaining: 198ms
446:	learn: 0.0073835	total: 920ms	remaining: 196ms
447:	learn: 0.0073630	total: 922ms	remaining: 193ms
448:	learn: 0.0073426	total: 924ms	remaining: 191ms
449:	learn: 0.0073426	total: 925ms	remaining: 189ms
450:	learn: 0.0073111	total: 927ms	remaining: 187ms
451:	learn: 0.0072952	total: 929ms	remaining: 185ms
452:	learn: 0.0072698	total: 931ms	remaining: 183ms
453:	learn: 0.0072317	total: 932ms	remaining: 181ms
454:	learn: 0.0072032	total: 935ms	remaining: 179ms
455:	learn: 0.0071686	total: 936ms	remaining: 177ms
456:	learn: 0.0071478	total: 938ms	remaining: 175ms
457:	learn: 0.0071478	total: 940ms	remaining: 172ms
458:	learn: 0.0071023	total: 942ms	remaining: 170ms
459:	learn: 0.0070819	total: 943ms	remaining: 168ms
460:	learn: 0.0070646	total: 945ms	remaining: 166ms
461:	learn: 0.0070647	total: 947ms	remaining: 164ms
462:	learn: 0.0070442	total: 949ms	remaining: 162ms
463:	learn: 0.0070201	total: 951ms	remaining: 160ms
464:	learn: 0.0069911	total: 953ms	remaining: 158ms
465:	learn: 0.0069578	total: 955ms	remaining: 156ms
466:	learn: 0.0069376	total: 956ms	remaining: 154ms
467:	learn: 0.0068997	total: 958ms	remaining: 152ms
468:	learn: 0.0068601	total: 960ms	remaining: 149ms
469:	learn: 0.0068430	total: 962ms	remaining: 147ms
470:	learn: 0.0068255	total: 963ms	remaining: 145ms
471:	learn: 0.0068086	total: 965ms	remaining: 143ms
472:	learn: 0.0067851	total: 967ms	remaining: 141ms
473:	learn: 0.0067682	total: 969ms	remaining: 139ms
474:	learn: 0.0067519	total: 970ms	remaining: 137ms
475:	learn: 0.0067348	total: 972ms	remaining: 135ms
476:	learn: 0.0067139	total: 974ms	remaining: 133ms
477:	learn: 0.0066855	total: 976ms	remaining: 131ms
478:	learn: 0.0066848	total: 977ms	remaining: 129ms
479:	learn: 0.0066681	total: 979ms	remaining: 126ms
480:	learn: 0.0066533	total: 980ms	remaining: 124ms
481:	learn: 0.0066351	total: 982ms	remaining: 122ms
482:	learn: 0.0066080	total: 984ms	remaining: 120ms
483:	learn: 0.0065904	total: 986ms	remaining: 118ms
484:	learn: 0.0065664	total: 987ms	remaining: 116ms
485:	learn: 0.0065645	total: 989ms	remaining: 114ms
486:	learn: 0.0065641	total: 991ms	remaining: 112ms
487:	learn: 0.0065414	total: 992ms	remaining: 110ms
488:	learn: 0.0065127	total: 994ms	remaining: 108ms
489:	learn: 0.0064996	total: 996ms	remaining: 106ms
490:	learn: 0.0064992	total: 997ms	remaining: 104ms
491:	learn: 0.0064779	total: 999ms	remaining: 101ms
492:	learn: 0.0064759	total: 1s	remaining: 99.4ms
493:	learn: 0.0064756	total: 1s	remaining: 97.3ms
494:	learn: 0.0064356	total: 1s	remaining: 95.3ms
495:	learn: 0.0064160	total: 1.01s	remaining: 93.3ms
496:	learn: 0.0063866	total: 1.01s	remaining: 91.3ms
497:	learn: 0.0063542	total: 1.01s	remaining: 89.3ms
498:	learn: 0.0063346	total: 1.01s	remaining: 87.3ms
499:	learn: 0.0063099	total: 1.01s	remaining: 85.3ms
500:	learn: 0.0062857	total: 1.02s	remaining: 83.2ms
501:	learn: 0.0062609	total: 1.02s	remaining: 81.2ms
502:	learn: 0.0062345	total: 1.02s	remaining: 79.2ms
503:	learn: 0.0062060	total: 1.02s	remaining: 77.2ms
504:	learn: 0.0061885	total: 1.02s	remaining: 75.1ms
505:	learn: 0.0061641	total: 1.03s	remaining: 73.1ms
506:	learn: 0.0061437	total: 1.03s	remaining: 71ms
507:	learn: 0.0061216	total: 1.03s	remaining: 69ms
508:	learn: 0.0060896	total: 1.03s	remaining: 66.9ms
509:	learn: 0.0060767	total: 1.03s	remaining: 64.9ms
510:	learn: 0.0060621	total: 1.03s	remaining: 62.8ms
511:	learn: 0.0060328	total: 1.04s	remaining: 60.8ms
512:	learn: 0.0060139	total: 1.04s	remaining: 58.8ms
513:	learn: 0.0060002	total: 1.04s	remaining: 56.8ms
514:	learn: 0.0059797	total: 1.04s	remaining: 54.7ms
515:	learn: 0.0059603	total: 1.04s	remaining: 52.7ms
516:	learn: 0.0059472	total: 1.05s	remaining: 50.7ms
517:	learn: 0.0059225	total: 1.05s	remaining: 48.6ms
518:	learn: 0.0059050	total: 1.05s	remaining: 46.6ms
519:	learn: 0.0058916	total: 1.05s	remaining: 44.5ms
520:	learn: 0.0058733	total: 1.05s	remaining: 42.5ms
521:	learn: 0.0058526	total: 1.06s	remaining: 40.5ms
522:	learn: 0.0058342	total: 1.06s	remaining: 38.4ms
523:	learn: 0.0058079	total: 1.06s	remaining: 36.4ms
524:	learn: 0.0057927	total: 1.06s	remaining: 34.4ms
525:	learn: 0.0057762	total: 1.06s	remaining: 32.4ms
526:	learn: 0.0057654	total: 1.06s	remaining: 30.3ms
527:	learn: 0.0057526	total: 1.07s	remaining: 28.3ms
528:	learn: 0.0057450	total: 1.07s	remaining: 26.3ms
529:	learn: 0.0057325	total: 1.07s	remaining: 24.2ms
530:	learn: 0.0057165	total: 1.07s	remaining: 22.2ms
531:	learn: 0.0057125	total: 1.07s	remaining: 20.2ms
532:	learn: 0.0056983	total: 1.08s	remaining: 18.2ms
533:	learn: 0.0056701	total: 1.08s	remaining: 16.2ms
534:	learn: 0.0056508	total: 1.08s	remaining: 14.1ms
535:	learn: 0.0056508	total: 1.08s	remaining: 12.1ms
536:	learn: 0.0056386	total: 1.08s	remaining: 10.1ms
537:	learn: 0.0056259	total: 1.08s	remaining: 8.07ms
538:	learn: 0.0056258	total: 1.09s	remaining: 6.05ms
539:	learn: 0.0056067	total: 1.09s	remaining: 4.03ms
540:	learn: 0.0055955	total: 1.09s	remaining: 2.02ms
541:	learn: 0.0055776	total: 1.09s	remaining: 0us
0:	learn: 0.5491296	total: 1.78ms	remaining: 963ms
1:	learn: 0.4718966	total: 3.03ms	remaining: 819ms
2:	learn: 0.4277265	total: 4.27ms	remaining: 768ms
3:	learn: 0.3757354	total: 5.9ms	remaining: 794ms
4:	learn: 0.3523291	total: 7.54ms	remaining: 810ms
5:	learn: 0.3294068	total: 9.26ms	remaining: 827ms
6:	learn: 0.3109888	total: 11ms	remaining: 844ms
7:	learn: 0.2969470	total: 12.8ms	remaining: 858ms
8:	learn: 0.2855056	total: 14.9ms	remaining: 880ms
9:	learn: 0.2739108	total: 16.5ms	remaining: 877ms
10:	learn: 0.2632521	total: 18.3ms	remaining: 882ms
11:	learn: 0.2514177	total: 20.1ms	remaining: 888ms
12:	learn: 0.2416608	total: 22ms	remaining: 894ms
13:	learn: 0.2232928	total: 23.8ms	remaining: 897ms
14:	learn: 0.2114778	total: 25.5ms	remaining: 895ms
15:	learn: 0.2058737	total: 27.4ms	remaining: 900ms
16:	learn: 0.2032343	total: 29.1ms	remaining: 899ms
17:	learn: 0.1938427	total: 30.7ms	remaining: 895ms
18:	learn: 0.1912491	total: 32.2ms	remaining: 887ms
19:	learn: 0.1866117	total: 33.7ms	remaining: 880ms
20:	learn: 0.1812460	total: 35.2ms	remaining: 874ms
21:	learn: 0.1759068	total: 36.7ms	remaining: 867ms
22:	learn: 0.1710826	total: 38.4ms	remaining: 865ms
23:	learn: 0.1689753	total: 40.2ms	remaining: 867ms
24:	learn: 0.1617038	total: 42.1ms	remaining: 870ms
25:	learn: 0.1574261	total: 44ms	remaining: 872ms
26:	learn: 0.1517412	total: 45.7ms	remaining: 872ms
27:	learn: 0.1449188	total: 47.4ms	remaining: 870ms
28:	learn: 0.1430510	total: 49ms	remaining: 866ms
29:	learn: 0.1369434	total: 50.5ms	remaining: 862ms
30:	learn: 0.1321353	total: 52ms	remaining: 858ms
31:	learn: 0.1278856	total: 53.5ms	remaining: 853ms
32:	learn: 0.1249358	total: 55ms	remaining: 848ms
33:	learn: 0.1225726	total: 56.7ms	remaining: 847ms
34:	learn: 0.1158340	total: 58.4ms	remaining: 845ms
35:	learn: 0.1112553	total: 59.9ms	remaining: 841ms
36:	learn: 0.1072678	total: 61.6ms	remaining: 840ms
37:	learn: 0.1058622	total: 63.1ms	remaining: 837ms
38:	learn: 0.1044869	total: 64.7ms	remaining: 834ms
39:	learn: 0.1011011	total: 66.2ms	remaining: 831ms
40:	learn: 0.0987301	total: 67.7ms	remaining: 827ms
41:	learn: 0.0954259	total: 69.3ms	remaining: 826ms
42:	learn: 0.0930467	total: 71ms	remaining: 824ms
43:	learn: 0.0914304	total: 72.8ms	remaining: 823ms
44:	learn: 0.0900781	total: 74.5ms	remaining: 823ms
45:	learn: 0.0863013	total: 76.5ms	remaining: 825ms
46:	learn: 0.0839285	total: 78.3ms	remaining: 824ms
47:	learn: 0.0806086	total: 79.8ms	remaining: 822ms
48:	learn: 0.0804716	total: 81.5ms	remaining: 820ms
49:	learn: 0.0783101	total: 83.3ms	remaining: 819ms
50:	learn: 0.0764151	total: 84.9ms	remaining: 818ms
51:	learn: 0.0747964	total: 86.6ms	remaining: 816ms
52:	learn: 0.0739089	total: 88.2ms	remaining: 813ms
53:	learn: 0.0738369	total: 89.7ms	remaining: 811ms
54:	learn: 0.0737663	total: 91.2ms	remaining: 808ms
55:	learn: 0.0736972	total: 92.8ms	remaining: 806ms
56:	learn: 0.0736866	total: 94.4ms	remaining: 803ms
57:	learn: 0.0714761	total: 96ms	remaining: 801ms
58:	learn: 0.0699014	total: 97.6ms	remaining: 799ms
59:	learn: 0.0681233	total: 99.1ms	remaining: 796ms
60:	learn: 0.0675716	total: 101ms	remaining: 793ms
61:	learn: 0.0662414	total: 102ms	remaining: 792ms
62:	learn: 0.0646889	total: 104ms	remaining: 791ms
63:	learn: 0.0631065	total: 106ms	remaining: 790ms
64:	learn: 0.0614332	total: 108ms	remaining: 789ms
65:	learn: 0.0601261	total: 109ms	remaining: 787ms
66:	learn: 0.0584377	total: 111ms	remaining: 785ms
67:	learn: 0.0573741	total: 112ms	remaining: 784ms
68:	learn: 0.0561980	total: 114ms	remaining: 783ms
69:	learn: 0.0549546	total: 116ms	remaining: 782ms
70:	learn: 0.0541912	total: 118ms	remaining: 781ms
71:	learn: 0.0531598	total: 119ms	remaining: 780ms
72:	learn: 0.0523873	total: 121ms	remaining: 778ms
73:	learn: 0.0511087	total: 123ms	remaining: 776ms
74:	learn: 0.0503143	total: 124ms	remaining: 774ms
75:	learn: 0.0495188	total: 126ms	remaining: 771ms
76:	learn: 0.0482214	total: 127ms	remaining: 770ms
77:	learn: 0.0475127	total: 129ms	remaining: 768ms
78:	learn: 0.0461248	total: 131ms	remaining: 767ms
79:	learn: 0.0457594	total: 133ms	remaining: 769ms
80:	learn: 0.0449370	total: 135ms	remaining: 770ms
81:	learn: 0.0447241	total: 137ms	remaining: 768ms
82:	learn: 0.0441965	total: 138ms	remaining: 765ms
83:	learn: 0.0441802	total: 140ms	remaining: 763ms
84:	learn: 0.0441522	total: 142ms	remaining: 761ms
85:	learn: 0.0441029	total: 143ms	remaining: 759ms
86:	learn: 0.0440698	total: 145ms	remaining: 756ms
87:	learn: 0.0438445	total: 146ms	remaining: 753ms
88:	learn: 0.0433479	total: 147ms	remaining: 750ms
89:	learn: 0.0425036	total: 149ms	remaining: 748ms
90:	learn: 0.0418176	total: 150ms	remaining: 746ms
91:	learn: 0.0414842	total: 152ms	remaining: 743ms
92:	learn: 0.0412886	total: 154ms	remaining: 741ms
93:	learn: 0.0406746	total: 155ms	remaining: 739ms
94:	learn: 0.0405632	total: 156ms	remaining: 736ms
95:	learn: 0.0401024	total: 158ms	remaining: 734ms
96:	learn: 0.0392080	total: 160ms	remaining: 733ms
97:	learn: 0.0385334	total: 161ms	remaining: 731ms
98:	learn: 0.0376887	total: 163ms	remaining: 730ms
99:	learn: 0.0372233	total: 165ms	remaining: 729ms
100:	learn: 0.0363925	total: 166ms	remaining: 727ms
101:	learn: 0.0357488	total: 168ms	remaining: 725ms
102:	learn: 0.0352021	total: 170ms	remaining: 723ms
103:	learn: 0.0350048	total: 172ms	remaining: 723ms
104:	learn: 0.0350048	total: 173ms	remaining: 722ms
105:	learn: 0.0349909	total: 175ms	remaining: 721ms
106:	learn: 0.0349884	total: 177ms	remaining: 720ms
107:	learn: 0.0349851	total: 179ms	remaining: 720ms
108:	learn: 0.0349825	total: 195ms	remaining: 774ms
109:	learn: 0.0349823	total: 197ms	remaining: 773ms
110:	learn: 0.0349811	total: 209ms	remaining: 811ms
111:	learn: 0.0344611	total: 210ms	remaining: 808ms
112:	learn: 0.0339228	total: 212ms	remaining: 806ms
113:	learn: 0.0333924	total: 214ms	remaining: 803ms
114:	learn: 0.0327784	total: 216ms	remaining: 801ms
115:	learn: 0.0325210	total: 217ms	remaining: 798ms
116:	learn: 0.0318409	total: 219ms	remaining: 796ms
117:	learn: 0.0315260	total: 221ms	remaining: 794ms
118:	learn: 0.0309106	total: 223ms	remaining: 793ms
119:	learn: 0.0303228	total: 225ms	remaining: 791ms
120:	learn: 0.0299666	total: 227ms	remaining: 789ms
121:	learn: 0.0299617	total: 229ms	remaining: 788ms
122:	learn: 0.0296251	total: 231ms	remaining: 786ms
123:	learn: 0.0292317	total: 233ms	remaining: 784ms
124:	learn: 0.0289466	total: 234ms	remaining: 782ms
125:	learn: 0.0283691	total: 236ms	remaining: 780ms
126:	learn: 0.0280975	total: 238ms	remaining: 778ms
127:	learn: 0.0276608	total: 240ms	remaining: 776ms
128:	learn: 0.0274120	total: 242ms	remaining: 773ms
129:	learn: 0.0272391	total: 243ms	remaining: 771ms
130:	learn: 0.0268395	total: 245ms	remaining: 768ms
131:	learn: 0.0268395	total: 246ms	remaining: 765ms
132:	learn: 0.0264406	total: 254ms	remaining: 780ms
133:	learn: 0.0260860	total: 256ms	remaining: 780ms
134:	learn: 0.0257333	total: 260ms	remaining: 783ms
135:	learn: 0.0256117	total: 262ms	remaining: 783ms
136:	learn: 0.0254458	total: 265ms	remaining: 783ms
137:	learn: 0.0251387	total: 300ms	remaining: 879ms
138:	learn: 0.0249568	total: 302ms	remaining: 876ms
139:	learn: 0.0249569	total: 304ms	remaining: 872ms
140:	learn: 0.0249568	total: 306ms	remaining: 869ms
141:	learn: 0.0249560	total: 307ms	remaining: 866ms
142:	learn: 0.0249559	total: 309ms	remaining: 863ms
143:	learn: 0.0249559	total: 311ms	remaining: 861ms
144:	learn: 0.0249551	total: 313ms	remaining: 858ms
145:	learn: 0.0249544	total: 315ms	remaining: 855ms
146:	learn: 0.0249537	total: 317ms	remaining: 852ms
147:	learn: 0.0249533	total: 319ms	remaining: 850ms
148:	learn: 0.0249529	total: 321ms	remaining: 846ms
149:	learn: 0.0249524	total: 323ms	remaining: 843ms
150:	learn: 0.0249524	total: 325ms	remaining: 841ms
151:	learn: 0.0249520	total: 327ms	remaining: 838ms
152:	learn: 0.0249518	total: 329ms	remaining: 835ms
153:	learn: 0.0249517	total: 341ms	remaining: 860ms
154:	learn: 0.0249516	total: 343ms	remaining: 856ms
155:	learn: 0.0249508	total: 345ms	remaining: 852ms
156:	learn: 0.0249491	total: 346ms	remaining: 849ms
157:	learn: 0.0249366	total: 348ms	remaining: 845ms
158:	learn: 0.0249357	total: 350ms	remaining: 842ms
159:	learn: 0.0249345	total: 351ms	remaining: 839ms
160:	learn: 0.0249345	total: 353ms	remaining: 835ms
161:	learn: 0.0249339	total: 354ms	remaining: 831ms
162:	learn: 0.0249336	total: 356ms	remaining: 828ms
163:	learn: 0.0249332	total: 357ms	remaining: 824ms
164:	learn: 0.0246075	total: 359ms	remaining: 821ms
165:	learn: 0.0241036	total: 361ms	remaining: 818ms
166:	learn: 0.0238748	total: 363ms	remaining: 815ms
167:	learn: 0.0237486	total: 365ms	remaining: 813ms
168:	learn: 0.0234308	total: 367ms	remaining: 810ms
169:	learn: 0.0231472	total: 372ms	remaining: 813ms
170:	learn: 0.0231379	total: 373ms	remaining: 810ms
171:	learn: 0.0231209	total: 375ms	remaining: 807ms
172:	learn: 0.0228700	total: 377ms	remaining: 804ms
173:	learn: 0.0227649	total: 378ms	remaining: 800ms
174:	learn: 0.0227507	total: 380ms	remaining: 797ms
175:	learn: 0.0227374	total: 381ms	remaining: 793ms
176:	learn: 0.0225011	total: 383ms	remaining: 790ms
177:	learn: 0.0222889	total: 385ms	remaining: 787ms
178:	learn: 0.0222704	total: 387ms	remaining: 785ms
179:	learn: 0.0219761	total: 389ms	remaining: 782ms
180:	learn: 0.0219610	total: 390ms	remaining: 779ms
181:	learn: 0.0219462	total: 392ms	remaining: 776ms
182:	learn: 0.0216641	total: 394ms	remaining: 773ms
183:	learn: 0.0216140	total: 396ms	remaining: 770ms
184:	learn: 0.0214181	total: 397ms	remaining: 767ms
185:	learn: 0.0214063	total: 400ms	remaining: 765ms
186:	learn: 0.0213933	total: 401ms	remaining: 762ms
187:	learn: 0.0213838	total: 403ms	remaining: 759ms
188:	learn: 0.0213420	total: 405ms	remaining: 756ms
189:	learn: 0.0210818	total: 407ms	remaining: 754ms
190:	learn: 0.0208486	total: 409ms	remaining: 751ms
191:	learn: 0.0206905	total: 410ms	remaining: 748ms
192:	learn: 0.0205355	total: 412ms	remaining: 745ms
193:	learn: 0.0204576	total: 414ms	remaining: 742ms
194:	learn: 0.0201879	total: 415ms	remaining: 739ms
195:	learn: 0.0199678	total: 417ms	remaining: 736ms
196:	learn: 0.0197145	total: 419ms	remaining: 733ms
197:	learn: 0.0196598	total: 420ms	remaining: 730ms
198:	learn: 0.0194725	total: 422ms	remaining: 728ms
199:	learn: 0.0193269	total: 424ms	remaining: 725ms
200:	learn: 0.0193107	total: 426ms	remaining: 722ms
201:	learn: 0.0190801	total: 427ms	remaining: 719ms
202:	learn: 0.0188806	total: 429ms	remaining: 716ms
203:	learn: 0.0188049	total: 431ms	remaining: 713ms
204:	learn: 0.0186403	total: 432ms	remaining: 710ms
205:	learn: 0.0184807	total: 434ms	remaining: 708ms
206:	learn: 0.0183060	total: 436ms	remaining: 705ms
207:	learn: 0.0182445	total: 437ms	remaining: 702ms
208:	learn: 0.0182360	total: 439ms	remaining: 699ms
209:	learn: 0.0182294	total: 440ms	remaining: 696ms
210:	learn: 0.0180758	total: 442ms	remaining: 693ms
211:	learn: 0.0179094	total: 443ms	remaining: 690ms
212:	learn: 0.0177155	total: 445ms	remaining: 688ms
213:	learn: 0.0177078	total: 447ms	remaining: 685ms
214:	learn: 0.0176991	total: 448ms	remaining: 682ms
215:	learn: 0.0175563	total: 450ms	remaining: 680ms
216:	learn: 0.0174377	total: 452ms	remaining: 677ms
217:	learn: 0.0174224	total: 454ms	remaining: 675ms
218:	learn: 0.0174225	total: 456ms	remaining: 672ms
219:	learn: 0.0174170	total: 458ms	remaining: 670ms
220:	learn: 0.0174091	total: 460ms	remaining: 668ms
221:	learn: 0.0172580	total: 462ms	remaining: 665ms
222:	learn: 0.0170730	total: 464ms	remaining: 663ms
223:	learn: 0.0170029	total: 466ms	remaining: 661ms
224:	learn: 0.0169036	total: 467ms	remaining: 658ms
225:	learn: 0.0167755	total: 469ms	remaining: 656ms
226:	learn: 0.0167707	total: 471ms	remaining: 653ms
227:	learn: 0.0166059	total: 472ms	remaining: 651ms
228:	learn: 0.0165699	total: 474ms	remaining: 648ms
229:	learn: 0.0164422	total: 476ms	remaining: 645ms
230:	learn: 0.0163482	total: 478ms	remaining: 643ms
231:	learn: 0.0161530	total: 479ms	remaining: 640ms
232:	learn: 0.0161311	total: 481ms	remaining: 638ms
233:	learn: 0.0159983	total: 483ms	remaining: 635ms
234:	learn: 0.0158391	total: 484ms	remaining: 633ms
235:	learn: 0.0157259	total: 486ms	remaining: 630ms
236:	learn: 0.0157175	total: 488ms	remaining: 628ms
237:	learn: 0.0155926	total: 490ms	remaining: 625ms
238:	learn: 0.0155099	total: 491ms	remaining: 623ms
239:	learn: 0.0154292	total: 493ms	remaining: 620ms
240:	learn: 0.0153671	total: 495ms	remaining: 618ms
241:	learn: 0.0153672	total: 496ms	remaining: 615ms
242:	learn: 0.0153648	total: 498ms	remaining: 613ms
243:	learn: 0.0153620	total: 499ms	remaining: 610ms
244:	learn: 0.0152467	total: 501ms	remaining: 607ms
245:	learn: 0.0151287	total: 503ms	remaining: 605ms
246:	learn: 0.0150424	total: 504ms	remaining: 602ms
247:	learn: 0.0149047	total: 506ms	remaining: 600ms
248:	learn: 0.0148030	total: 508ms	remaining: 597ms
249:	learn: 0.0147018	total: 509ms	remaining: 595ms
250:	learn: 0.0146728	total: 511ms	remaining: 593ms
251:	learn: 0.0145546	total: 514ms	remaining: 591ms
252:	learn: 0.0145108	total: 516ms	remaining: 589ms
253:	learn: 0.0144472	total: 518ms	remaining: 587ms
254:	learn: 0.0144129	total: 520ms	remaining: 585ms
255:	learn: 0.0143812	total: 522ms	remaining: 583ms
256:	learn: 0.0142305	total: 523ms	remaining: 580ms
257:	learn: 0.0142285	total: 525ms	remaining: 578ms
258:	learn: 0.0142284	total: 526ms	remaining: 575ms
259:	learn: 0.0142263	total: 528ms	remaining: 573ms
260:	learn: 0.0141949	total: 530ms	remaining: 570ms
261:	learn: 0.0141501	total: 532ms	remaining: 568ms
262:	learn: 0.0141471	total: 533ms	remaining: 566ms
263:	learn: 0.0141446	total: 535ms	remaining: 563ms
264:	learn: 0.0141424	total: 537ms	remaining: 561ms
265:	learn: 0.0140679	total: 538ms	remaining: 559ms
266:	learn: 0.0140048	total: 540ms	remaining: 556ms
267:	learn: 0.0139781	total: 542ms	remaining: 554ms
268:	learn: 0.0138651	total: 544ms	remaining: 552ms
269:	learn: 0.0137599	total: 546ms	remaining: 550ms
270:	learn: 0.0136736	total: 548ms	remaining: 548ms
271:	learn: 0.0135674	total: 550ms	remaining: 546ms
272:	learn: 0.0135407	total: 552ms	remaining: 544ms
273:	learn: 0.0134608	total: 554ms	remaining: 542ms
274:	learn: 0.0133565	total: 557ms	remaining: 540ms
275:	learn: 0.0132149	total: 559ms	remaining: 538ms
276:	learn: 0.0132137	total: 561ms	remaining: 536ms
277:	learn: 0.0132066	total: 562ms	remaining: 534ms
278:	learn: 0.0132066	total: 564ms	remaining: 531ms
279:	learn: 0.0132066	total: 565ms	remaining: 529ms
280:	learn: 0.0132065	total: 567ms	remaining: 526ms
281:	learn: 0.0132065	total: 568ms	remaining: 524ms
282:	learn: 0.0132064	total: 570ms	remaining: 521ms
283:	learn: 0.0132065	total: 571ms	remaining: 519ms
284:	learn: 0.0132065	total: 573ms	remaining: 516ms
285:	learn: 0.0132065	total: 574ms	remaining: 514ms
286:	learn: 0.0132064	total: 576ms	remaining: 512ms
287:	learn: 0.0132063	total: 577ms	remaining: 509ms
288:	learn: 0.0132063	total: 579ms	remaining: 507ms
289:	learn: 0.0132061	total: 581ms	remaining: 505ms
290:	learn: 0.0132061	total: 582ms	remaining: 502ms
291:	learn: 0.0132061	total: 583ms	remaining: 500ms
292:	learn: 0.0131138	total: 585ms	remaining: 497ms
293:	learn: 0.0130384	total: 587ms	remaining: 495ms
294:	learn: 0.0129699	total: 588ms	remaining: 493ms
295:	learn: 0.0128648	total: 590ms	remaining: 490ms
296:	learn: 0.0128648	total: 591ms	remaining: 488ms
297:	learn: 0.0127802	total: 593ms	remaining: 486ms
298:	learn: 0.0127786	total: 595ms	remaining: 483ms
299:	learn: 0.0127776	total: 596ms	remaining: 481ms
300:	learn: 0.0127774	total: 598ms	remaining: 479ms
301:	learn: 0.0127774	total: 600ms	remaining: 476ms
302:	learn: 0.0127774	total: 602ms	remaining: 475ms
303:	learn: 0.0127773	total: 604ms	remaining: 472ms
304:	learn: 0.0127773	total: 605ms	remaining: 470ms
305:	learn: 0.0127773	total: 607ms	remaining: 468ms
306:	learn: 0.0127771	total: 609ms	remaining: 466ms
307:	learn: 0.0127772	total: 610ms	remaining: 464ms
308:	learn: 0.0127769	total: 612ms	remaining: 462ms
309:	learn: 0.0127769	total: 614ms	remaining: 459ms
310:	learn: 0.0127770	total: 616ms	remaining: 457ms
311:	learn: 0.0127769	total: 617ms	remaining: 455ms
312:	learn: 0.0127768	total: 619ms	remaining: 453ms
313:	learn: 0.0127768	total: 620ms	remaining: 451ms
314:	learn: 0.0127766	total: 622ms	remaining: 448ms
315:	learn: 0.0127766	total: 624ms	remaining: 446ms
316:	learn: 0.0127765	total: 625ms	remaining: 444ms
317:	learn: 0.0127764	total: 627ms	remaining: 441ms
318:	learn: 0.0127763	total: 628ms	remaining: 439ms
319:	learn: 0.0127763	total: 630ms	remaining: 437ms
320:	learn: 0.0127762	total: 632ms	remaining: 435ms
321:	learn: 0.0127762	total: 634ms	remaining: 433ms
322:	learn: 0.0127761	total: 636ms	remaining: 431ms
323:	learn: 0.0127761	total: 637ms	remaining: 429ms
324:	learn: 0.0127761	total: 639ms	remaining: 427ms
325:	learn: 0.0127761	total: 641ms	remaining: 424ms
326:	learn: 0.0127761	total: 642ms	remaining: 422ms
327:	learn: 0.0127760	total: 644ms	remaining: 420ms
328:	learn: 0.0127760	total: 646ms	remaining: 418ms
329:	learn: 0.0127758	total: 647ms	remaining: 416ms
330:	learn: 0.0127757	total: 649ms	remaining: 413ms
331:	learn: 0.0127755	total: 650ms	remaining: 411ms
332:	learn: 0.0127752	total: 652ms	remaining: 409ms
333:	learn: 0.0127751	total: 653ms	remaining: 407ms
334:	learn: 0.0127751	total: 655ms	remaining: 404ms
335:	learn: 0.0127750	total: 656ms	remaining: 402ms
336:	learn: 0.0127749	total: 658ms	remaining: 400ms
337:	learn: 0.0127749	total: 660ms	remaining: 398ms
338:	learn: 0.0126858	total: 662ms	remaining: 396ms
339:	learn: 0.0126857	total: 663ms	remaining: 394ms
340:	learn: 0.0126857	total: 665ms	remaining: 392ms
341:	learn: 0.0126856	total: 666ms	remaining: 390ms
342:	learn: 0.0126856	total: 668ms	remaining: 387ms
343:	learn: 0.0126857	total: 669ms	remaining: 385ms
344:	learn: 0.0126854	total: 671ms	remaining: 383ms
345:	learn: 0.0126855	total: 672ms	remaining: 381ms
346:	learn: 0.0126853	total: 673ms	remaining: 378ms
347:	learn: 0.0126853	total: 675ms	remaining: 376ms
348:	learn: 0.0126854	total: 676ms	remaining: 374ms
349:	learn: 0.0126851	total: 678ms	remaining: 372ms
350:	learn: 0.0126850	total: 679ms	remaining: 370ms
351:	learn: 0.0126850	total: 681ms	remaining: 367ms
352:	learn: 0.0126850	total: 682ms	remaining: 365ms
353:	learn: 0.0126849	total: 684ms	remaining: 363ms
354:	learn: 0.0126848	total: 685ms	remaining: 361ms
355:	learn: 0.0126848	total: 687ms	remaining: 359ms
356:	learn: 0.0126846	total: 688ms	remaining: 357ms
357:	learn: 0.0126845	total: 690ms	remaining: 354ms
358:	learn: 0.0126845	total: 691ms	remaining: 352ms
359:	learn: 0.0126845	total: 693ms	remaining: 350ms
360:	learn: 0.0126843	total: 694ms	remaining: 348ms
361:	learn: 0.0126843	total: 695ms	remaining: 346ms
362:	learn: 0.0126843	total: 697ms	remaining: 344ms
363:	learn: 0.0126842	total: 698ms	remaining: 341ms
364:	learn: 0.0126843	total: 700ms	remaining: 339ms
365:	learn: 0.0126842	total: 701ms	remaining: 337ms
366:	learn: 0.0126842	total: 702ms	remaining: 335ms
367:	learn: 0.0126842	total: 704ms	remaining: 333ms
368:	learn: 0.0126842	total: 706ms	remaining: 331ms
369:	learn: 0.0126841	total: 708ms	remaining: 329ms
370:	learn: 0.0126840	total: 709ms	remaining: 327ms
371:	learn: 0.0126839	total: 711ms	remaining: 325ms
372:	learn: 0.0126839	total: 712ms	remaining: 323ms
373:	learn: 0.0126839	total: 714ms	remaining: 321ms
374:	learn: 0.0126838	total: 715ms	remaining: 318ms
375:	learn: 0.0126838	total: 717ms	remaining: 316ms
376:	learn: 0.0126838	total: 718ms	remaining: 314ms
377:	learn: 0.0126836	total: 720ms	remaining: 312ms
378:	learn: 0.0126835	total: 721ms	remaining: 310ms
379:	learn: 0.0126835	total: 723ms	remaining: 308ms
380:	learn: 0.0126835	total: 724ms	remaining: 306ms
381:	learn: 0.0126834	total: 726ms	remaining: 304ms
382:	learn: 0.0126833	total: 727ms	remaining: 302ms
383:	learn: 0.0126833	total: 729ms	remaining: 300ms
384:	learn: 0.0126831	total: 730ms	remaining: 298ms
385:	learn: 0.0126830	total: 731ms	remaining: 296ms
386:	learn: 0.0126830	total: 733ms	remaining: 294ms
387:	learn: 0.0126830	total: 735ms	remaining: 292ms
388:	learn: 0.0126830	total: 737ms	remaining: 290ms
389:	learn: 0.0126830	total: 739ms	remaining: 288ms
390:	learn: 0.0126830	total: 740ms	remaining: 286ms
391:	learn: 0.0126829	total: 742ms	remaining: 284ms
392:	learn: 0.0126829	total: 743ms	remaining: 282ms
393:	learn: 0.0126829	total: 758ms	remaining: 285ms
394:	learn: 0.0126828	total: 760ms	remaining: 283ms
395:	learn: 0.0126828	total: 761ms	remaining: 281ms
396:	learn: 0.0126049	total: 763ms	remaining: 279ms
397:	learn: 0.0125205	total: 765ms	remaining: 277ms
398:	learn: 0.0124231	total: 767ms	remaining: 275ms
399:	learn: 0.0123551	total: 769ms	remaining: 273ms
400:	learn: 0.0122768	total: 770ms	remaining: 271ms
401:	learn: 0.0122259	total: 772ms	remaining: 269ms
402:	learn: 0.0121716	total: 774ms	remaining: 267ms
403:	learn: 0.0121081	total: 775ms	remaining: 265ms
404:	learn: 0.0121073	total: 777ms	remaining: 263ms
405:	learn: 0.0121073	total: 778ms	remaining: 261ms
406:	learn: 0.0121073	total: 780ms	remaining: 259ms
407:	learn: 0.0121072	total: 782ms	remaining: 257ms
408:	learn: 0.0121072	total: 783ms	remaining: 255ms
409:	learn: 0.0121072	total: 784ms	remaining: 253ms
410:	learn: 0.0121072	total: 786ms	remaining: 250ms
411:	learn: 0.0121071	total: 787ms	remaining: 248ms
412:	learn: 0.0121070	total: 789ms	remaining: 246ms
413:	learn: 0.0121070	total: 790ms	remaining: 244ms
414:	learn: 0.0121070	total: 791ms	remaining: 242ms
415:	learn: 0.0121069	total: 793ms	remaining: 240ms
416:	learn: 0.0121069	total: 794ms	remaining: 238ms
417:	learn: 0.0121068	total: 796ms	remaining: 236ms
418:	learn: 0.0121069	total: 798ms	remaining: 234ms
419:	learn: 0.0121067	total: 799ms	remaining: 232ms
420:	learn: 0.0121066	total: 801ms	remaining: 230ms
421:	learn: 0.0121066	total: 803ms	remaining: 228ms
422:	learn: 0.0121066	total: 804ms	remaining: 226ms
423:	learn: 0.0121065	total: 805ms	remaining: 224ms
424:	learn: 0.0121065	total: 807ms	remaining: 222ms
425:	learn: 0.0121064	total: 808ms	remaining: 220ms
426:	learn: 0.0121063	total: 809ms	remaining: 218ms
427:	learn: 0.0121063	total: 811ms	remaining: 216ms
428:	learn: 0.0121062	total: 812ms	remaining: 214ms
429:	learn: 0.0121062	total: 814ms	remaining: 212ms
430:	learn: 0.0121062	total: 816ms	remaining: 210ms
431:	learn: 0.0121061	total: 818ms	remaining: 208ms
432:	learn: 0.0121062	total: 820ms	remaining: 206ms
433:	learn: 0.0121062	total: 821ms	remaining: 204ms
434:	learn: 0.0121060	total: 823ms	remaining: 202ms
435:	learn: 0.0121060	total: 824ms	remaining: 200ms
436:	learn: 0.0121058	total: 826ms	remaining: 198ms
437:	learn: 0.0121057	total: 828ms	remaining: 197ms
438:	learn: 0.0121057	total: 830ms	remaining: 195ms
439:	learn: 0.0121057	total: 832ms	remaining: 193ms
440:	learn: 0.0121057	total: 833ms	remaining: 191ms
441:	learn: 0.0121055	total: 835ms	remaining: 189ms
442:	learn: 0.0121053	total: 837ms	remaining: 187ms
443:	learn: 0.0120877	total: 839ms	remaining: 185ms
444:	learn: 0.0120877	total: 840ms	remaining: 183ms
445:	learn: 0.0120876	total: 842ms	remaining: 181ms
446:	learn: 0.0120876	total: 843ms	remaining: 179ms
447:	learn: 0.0120876	total: 845ms	remaining: 177ms
448:	learn: 0.0120876	total: 846ms	remaining: 175ms
449:	learn: 0.0120875	total: 848ms	remaining: 173ms
450:	learn: 0.0120875	total: 849ms	remaining: 171ms
451:	learn: 0.0120875	total: 851ms	remaining: 169ms
452:	learn: 0.0120874	total: 852ms	remaining: 167ms
453:	learn: 0.0120874	total: 854ms	remaining: 165ms
454:	learn: 0.0120874	total: 855ms	remaining: 164ms
455:	learn: 0.0120872	total: 857ms	remaining: 162ms
456:	learn: 0.0120872	total: 859ms	remaining: 160ms
457:	learn: 0.0120872	total: 860ms	remaining: 158ms
458:	learn: 0.0120872	total: 862ms	remaining: 156ms
459:	learn: 0.0120871	total: 864ms	remaining: 154ms
460:	learn: 0.0120218	total: 865ms	remaining: 152ms
461:	learn: 0.0118812	total: 867ms	remaining: 150ms
462:	learn: 0.0118033	total: 869ms	remaining: 148ms
463:	learn: 0.0117209	total: 871ms	remaining: 146ms
464:	learn: 0.0117201	total: 874ms	remaining: 145ms
465:	learn: 0.0117201	total: 876ms	remaining: 143ms
466:	learn: 0.0117200	total: 878ms	remaining: 141ms
467:	learn: 0.0117200	total: 880ms	remaining: 139ms
468:	learn: 0.0117199	total: 882ms	remaining: 137ms
469:	learn: 0.0117199	total: 884ms	remaining: 135ms
470:	learn: 0.0117198	total: 888ms	remaining: 134ms
471:	learn: 0.0117197	total: 890ms	remaining: 132ms
472:	learn: 0.0117197	total: 892ms	remaining: 130ms
473:	learn: 0.0117197	total: 894ms	remaining: 128ms
474:	learn: 0.0117197	total: 896ms	remaining: 126ms
475:	learn: 0.0117196	total: 898ms	remaining: 124ms
476:	learn: 0.0117195	total: 900ms	remaining: 123ms
477:	learn: 0.0117196	total: 902ms	remaining: 121ms
478:	learn: 0.0117195	total: 904ms	remaining: 119ms
479:	learn: 0.0117194	total: 906ms	remaining: 117ms
480:	learn: 0.0117194	total: 908ms	remaining: 115ms
481:	learn: 0.0117194	total: 909ms	remaining: 113ms
482:	learn: 0.0117192	total: 911ms	remaining: 111ms
483:	learn: 0.0117192	total: 912ms	remaining: 109ms
484:	learn: 0.0117192	total: 914ms	remaining: 107ms
485:	learn: 0.0117192	total: 915ms	remaining: 105ms
486:	learn: 0.0117193	total: 917ms	remaining: 104ms
487:	learn: 0.0116500	total: 919ms	remaining: 102ms
488:	learn: 0.0116500	total: 921ms	remaining: 99.8ms
489:	learn: 0.0116500	total: 923ms	remaining: 97.9ms
490:	learn: 0.0116500	total: 924ms	remaining: 96ms
491:	learn: 0.0116500	total: 926ms	remaining: 94.1ms
492:	learn: 0.0116499	total: 928ms	remaining: 92.2ms
493:	learn: 0.0116498	total: 930ms	remaining: 90.3ms
494:	learn: 0.0116498	total: 931ms	remaining: 88.4ms
495:	learn: 0.0116497	total: 933ms	remaining: 86.5ms
496:	learn: 0.0116496	total: 934ms	remaining: 84.6ms
497:	learn: 0.0116496	total: 936ms	remaining: 82.7ms
498:	learn: 0.0116496	total: 938ms	remaining: 80.8ms
499:	learn: 0.0116496	total: 940ms	remaining: 78.9ms
500:	learn: 0.0116495	total: 941ms	remaining: 77ms
501:	learn: 0.0116494	total: 943ms	remaining: 75.1ms
502:	learn: 0.0116494	total: 944ms	remaining: 73.2ms
503:	learn: 0.0116494	total: 946ms	remaining: 71.3ms
504:	learn: 0.0116494	total: 948ms	remaining: 69.4ms
505:	learn: 0.0115680	total: 949ms	remaining: 67.6ms
506:	learn: 0.0115680	total: 951ms	remaining: 65.6ms
507:	learn: 0.0115344	total: 952ms	remaining: 63.7ms
508:	learn: 0.0115330	total: 954ms	remaining: 61.8ms
509:	learn: 0.0115329	total: 955ms	remaining: 59.9ms
510:	learn: 0.0115329	total: 956ms	remaining: 58ms
511:	learn: 0.0115329	total: 958ms	remaining: 56.1ms
512:	learn: 0.0115327	total: 959ms	remaining: 54.2ms
513:	learn: 0.0115327	total: 961ms	remaining: 52.3ms
514:	learn: 0.0115326	total: 962ms	remaining: 50.4ms
515:	learn: 0.0115325	total: 963ms	remaining: 48.5ms
516:	learn: 0.0115324	total: 965ms	remaining: 46.6ms
517:	learn: 0.0115324	total: 966ms	remaining: 44.8ms
518:	learn: 0.0115322	total: 967ms	remaining: 42.9ms
519:	learn: 0.0115301	total: 969ms	remaining: 41ms
520:	learn: 0.0114786	total: 971ms	remaining: 39.1ms
521:	learn: 0.0114772	total: 972ms	remaining: 37.3ms
522:	learn: 0.0114772	total: 974ms	remaining: 35.4ms
523:	learn: 0.0114772	total: 975ms	remaining: 33.5ms
524:	learn: 0.0114771	total: 977ms	remaining: 31.6ms
525:	learn: 0.0114771	total: 978ms	remaining: 29.8ms
526:	learn: 0.0114770	total: 980ms	remaining: 27.9ms
527:	learn: 0.0114769	total: 982ms	remaining: 26ms
528:	learn: 0.0114768	total: 983ms	remaining: 24.2ms
529:	learn: 0.0114767	total: 985ms	remaining: 22.3ms
530:	learn: 0.0114766	total: 986ms	remaining: 20.4ms
531:	learn: 0.0114766	total: 988ms	remaining: 18.6ms
532:	learn: 0.0114767	total: 989ms	remaining: 16.7ms
533:	learn: 0.0114766	total: 991ms	remaining: 14.8ms
534:	learn: 0.0114764	total: 992ms	remaining: 13ms
535:	learn: 0.0114765	total: 993ms	remaining: 11.1ms
536:	learn: 0.0114765	total: 995ms	remaining: 9.26ms
537:	learn: 0.0114765	total: 996ms	remaining: 7.41ms
538:	learn: 0.0114764	total: 998ms	remaining: 5.55ms
539:	learn: 0.0114762	total: 999ms	remaining: 3.7ms
540:	learn: 0.0114762	total: 1s	remaining: 1.85ms
541:	learn: 0.0114762	total: 1s	remaining: 0us
0:	learn: 0.5608792	total: 1.89ms	remaining: 1.02s
1:	learn: 0.4739313	total: 3.66ms	remaining: 988ms
2:	learn: 0.4061240	total: 5.74ms	remaining: 1.03s
3:	learn: 0.3639638	total: 7.28ms	remaining: 979ms
4:	learn: 0.3329878	total: 9.32ms	remaining: 1s
5:	learn: 0.3016934	total: 12.3ms	remaining: 1.1s
6:	learn: 0.2775180	total: 14.4ms	remaining: 1.1s
7:	learn: 0.2574487	total: 16.4ms	remaining: 1.1s
8:	learn: 0.2439048	total: 18.2ms	remaining: 1.08s
9:	learn: 0.2358456	total: 20ms	remaining: 1.06s
10:	learn: 0.2245165	total: 21.8ms	remaining: 1.05s
11:	learn: 0.2161144	total: 23.6ms	remaining: 1.04s
12:	learn: 0.2076905	total: 25.4ms	remaining: 1.03s
13:	learn: 0.1986645	total: 27.3ms	remaining: 1.03s
14:	learn: 0.1953091	total: 29.2ms	remaining: 1.02s
15:	learn: 0.1882308	total: 30.9ms	remaining: 1.01s
16:	learn: 0.1845039	total: 32.7ms	remaining: 1.01s
17:	learn: 0.1817585	total: 34.4ms	remaining: 1s
18:	learn: 0.1745170	total: 36.3ms	remaining: 999ms
19:	learn: 0.1712011	total: 38.2ms	remaining: 996ms
20:	learn: 0.1677930	total: 40.2ms	remaining: 996ms
21:	learn: 0.1645778	total: 42ms	remaining: 993ms
22:	learn: 0.1601335	total: 43.9ms	remaining: 992ms
23:	learn: 0.1559169	total: 45.8ms	remaining: 989ms
24:	learn: 0.1552704	total: 47.9ms	remaining: 991ms
25:	learn: 0.1512472	total: 49.8ms	remaining: 988ms
26:	learn: 0.1477779	total: 51.7ms	remaining: 986ms
27:	learn: 0.1456297	total: 53.4ms	remaining: 981ms
28:	learn: 0.1405299	total: 55.4ms	remaining: 980ms
29:	learn: 0.1382775	total: 57.2ms	remaining: 976ms
30:	learn: 0.1363517	total: 58.9ms	remaining: 970ms
31:	learn: 0.1343982	total: 60.5ms	remaining: 965ms
32:	learn: 0.1336558	total: 62.3ms	remaining: 960ms
33:	learn: 0.1333923	total: 64ms	remaining: 957ms
34:	learn: 0.1294404	total: 65.7ms	remaining: 951ms
35:	learn: 0.1266193	total: 67.5ms	remaining: 949ms
36:	learn: 0.1241272	total: 69.3ms	remaining: 946ms
37:	learn: 0.1207207	total: 71.3ms	remaining: 946ms
38:	learn: 0.1196566	total: 72.9ms	remaining: 941ms
39:	learn: 0.1164913	total: 74.5ms	remaining: 935ms
40:	learn: 0.1143950	total: 76.1ms	remaining: 930ms
41:	learn: 0.1125190	total: 77.9ms	remaining: 927ms
42:	learn: 0.1110453	total: 79.8ms	remaining: 926ms
43:	learn: 0.1097767	total: 81.6ms	remaining: 923ms
44:	learn: 0.1093547	total: 83.3ms	remaining: 920ms
45:	learn: 0.1067977	total: 85.2ms	remaining: 919ms
46:	learn: 0.1035013	total: 87.1ms	remaining: 917ms
47:	learn: 0.1021569	total: 88.9ms	remaining: 915ms
48:	learn: 0.0992679	total: 90.7ms	remaining: 912ms
49:	learn: 0.0982974	total: 92.6ms	remaining: 911ms
50:	learn: 0.0962521	total: 94.8ms	remaining: 912ms
51:	learn: 0.0925672	total: 96.7ms	remaining: 911ms
52:	learn: 0.0900548	total: 98.9ms	remaining: 912ms
53:	learn: 0.0863702	total: 101ms	remaining: 909ms
54:	learn: 0.0833439	total: 102ms	remaining: 906ms
55:	learn: 0.0829930	total: 104ms	remaining: 903ms
56:	learn: 0.0813576	total: 106ms	remaining: 900ms
57:	learn: 0.0811267	total: 112ms	remaining: 933ms
58:	learn: 0.0809168	total: 114ms	remaining: 931ms
59:	learn: 0.0789429	total: 116ms	remaining: 929ms
60:	learn: 0.0787403	total: 118ms	remaining: 929ms
61:	learn: 0.0779670	total: 120ms	remaining: 929ms
62:	learn: 0.0755021	total: 122ms	remaining: 927ms
63:	learn: 0.0749543	total: 124ms	remaining: 925ms
64:	learn: 0.0741304	total: 126ms	remaining: 925ms
65:	learn: 0.0726814	total: 128ms	remaining: 923ms
66:	learn: 0.0708597	total: 130ms	remaining: 921ms
67:	learn: 0.0698767	total: 132ms	remaining: 917ms
68:	learn: 0.0680085	total: 133ms	remaining: 915ms
69:	learn: 0.0664510	total: 135ms	remaining: 912ms
70:	learn: 0.0650658	total: 137ms	remaining: 910ms
71:	learn: 0.0630875	total: 139ms	remaining: 907ms
72:	learn: 0.0619367	total: 141ms	remaining: 906ms
73:	learn: 0.0606144	total: 143ms	remaining: 904ms
74:	learn: 0.0596356	total: 145ms	remaining: 902ms
75:	learn: 0.0590030	total: 147ms	remaining: 900ms
76:	learn: 0.0576470	total: 149ms	remaining: 897ms
77:	learn: 0.0574494	total: 150ms	remaining: 894ms
78:	learn: 0.0565003	total: 152ms	remaining: 892ms
79:	learn: 0.0562553	total: 154ms	remaining: 891ms
80:	learn: 0.0558951	total: 156ms	remaining: 890ms
81:	learn: 0.0557654	total: 158ms	remaining: 888ms
82:	learn: 0.0546668	total: 160ms	remaining: 886ms
83:	learn: 0.0535102	total: 162ms	remaining: 885ms
84:	learn: 0.0532951	total: 164ms	remaining: 883ms
85:	learn: 0.0525262	total: 166ms	remaining: 882ms
86:	learn: 0.0517893	total: 168ms	remaining: 881ms
87:	learn: 0.0514096	total: 171ms	remaining: 880ms
88:	learn: 0.0506449	total: 173ms	remaining: 880ms
89:	learn: 0.0495428	total: 175ms	remaining: 879ms
90:	learn: 0.0490028	total: 177ms	remaining: 876ms
91:	learn: 0.0478885	total: 179ms	remaining: 873ms
92:	learn: 0.0470022	total: 180ms	remaining: 871ms
93:	learn: 0.0467709	total: 182ms	remaining: 867ms
94:	learn: 0.0465191	total: 183ms	remaining: 863ms
95:	learn: 0.0462991	total: 185ms	remaining: 860ms
96:	learn: 0.0461283	total: 187ms	remaining: 857ms
97:	learn: 0.0459496	total: 188ms	remaining: 854ms
98:	learn: 0.0450233	total: 190ms	remaining: 852ms
99:	learn: 0.0439916	total: 192ms	remaining: 850ms
100:	learn: 0.0430411	total: 194ms	remaining: 847ms
101:	learn: 0.0428566	total: 196ms	remaining: 845ms
102:	learn: 0.0422337	total: 198ms	remaining: 843ms
103:	learn: 0.0412971	total: 199ms	remaining: 840ms
104:	learn: 0.0411918	total: 201ms	remaining: 838ms
105:	learn: 0.0410726	total: 203ms	remaining: 836ms
106:	learn: 0.0409668	total: 205ms	remaining: 834ms
107:	learn: 0.0406157	total: 208ms	remaining: 834ms
108:	learn: 0.0401002	total: 210ms	remaining: 833ms
109:	learn: 0.0398019	total: 212ms	remaining: 832ms
110:	learn: 0.0397127	total: 214ms	remaining: 830ms
111:	learn: 0.0396162	total: 216ms	remaining: 829ms
112:	learn: 0.0394288	total: 218ms	remaining: 827ms
113:	learn: 0.0388875	total: 220ms	remaining: 825ms
114:	learn: 0.0383211	total: 221ms	remaining: 822ms
115:	learn: 0.0378935	total: 223ms	remaining: 820ms
116:	learn: 0.0374222	total: 240ms	remaining: 871ms
117:	learn: 0.0368531	total: 241ms	remaining: 867ms
118:	learn: 0.0361009	total: 243ms	remaining: 863ms
119:	learn: 0.0356156	total: 244ms	remaining: 860ms
120:	learn: 0.0351396	total: 246ms	remaining: 856ms
121:	learn: 0.0347406	total: 248ms	remaining: 854ms
122:	learn: 0.0339655	total: 250ms	remaining: 850ms
123:	learn: 0.0335403	total: 251ms	remaining: 847ms
124:	learn: 0.0331121	total: 253ms	remaining: 844ms
125:	learn: 0.0327770	total: 255ms	remaining: 841ms
126:	learn: 0.0323517	total: 256ms	remaining: 838ms
127:	learn: 0.0318561	total: 258ms	remaining: 835ms
128:	learn: 0.0317953	total: 260ms	remaining: 832ms
129:	learn: 0.0314416	total: 262ms	remaining: 829ms
130:	learn: 0.0311333	total: 263ms	remaining: 826ms
131:	learn: 0.0309637	total: 265ms	remaining: 824ms
132:	learn: 0.0305932	total: 268ms	remaining: 823ms
133:	learn: 0.0302134	total: 269ms	remaining: 820ms
134:	learn: 0.0301109	total: 271ms	remaining: 817ms
135:	learn: 0.0299303	total: 273ms	remaining: 814ms
136:	learn: 0.0295968	total: 275ms	remaining: 812ms
137:	learn: 0.0295238	total: 276ms	remaining: 809ms
138:	learn: 0.0291377	total: 278ms	remaining: 806ms
139:	learn: 0.0287741	total: 280ms	remaining: 803ms
140:	learn: 0.0286043	total: 281ms	remaining: 800ms
141:	learn: 0.0285333	total: 283ms	remaining: 797ms
142:	learn: 0.0284940	total: 285ms	remaining: 794ms
143:	learn: 0.0281941	total: 286ms	remaining: 791ms
144:	learn: 0.0281318	total: 288ms	remaining: 788ms
145:	learn: 0.0279864	total: 289ms	remaining: 784ms
146:	learn: 0.0279527	total: 291ms	remaining: 781ms
147:	learn: 0.0277950	total: 292ms	remaining: 779ms
148:	learn: 0.0277654	total: 295ms	remaining: 777ms
149:	learn: 0.0275901	total: 297ms	remaining: 775ms
150:	learn: 0.0271665	total: 299ms	remaining: 773ms
151:	learn: 0.0268400	total: 300ms	remaining: 770ms
152:	learn: 0.0265937	total: 302ms	remaining: 768ms
153:	learn: 0.0264309	total: 304ms	remaining: 765ms
154:	learn: 0.0261046	total: 306ms	remaining: 764ms
155:	learn: 0.0258538	total: 308ms	remaining: 761ms
156:	learn: 0.0257012	total: 309ms	remaining: 759ms
157:	learn: 0.0256647	total: 311ms	remaining: 756ms
158:	learn: 0.0256238	total: 313ms	remaining: 754ms
159:	learn: 0.0254097	total: 315ms	remaining: 751ms
160:	learn: 0.0252101	total: 317ms	remaining: 749ms
161:	learn: 0.0249955	total: 318ms	remaining: 746ms
162:	learn: 0.0249355	total: 320ms	remaining: 743ms
163:	learn: 0.0247721	total: 321ms	remaining: 741ms
164:	learn: 0.0246091	total: 323ms	remaining: 738ms
165:	learn: 0.0244730	total: 325ms	remaining: 736ms
166:	learn: 0.0242757	total: 327ms	remaining: 734ms
167:	learn: 0.0240714	total: 328ms	remaining: 731ms
168:	learn: 0.0237728	total: 330ms	remaining: 728ms
169:	learn: 0.0235921	total: 332ms	remaining: 726ms
170:	learn: 0.0233635	total: 333ms	remaining: 723ms
171:	learn: 0.0232864	total: 335ms	remaining: 720ms
172:	learn: 0.0231527	total: 336ms	remaining: 717ms
173:	learn: 0.0229906	total: 338ms	remaining: 715ms
174:	learn: 0.0228632	total: 340ms	remaining: 712ms
175:	learn: 0.0228328	total: 341ms	remaining: 710ms
176:	learn: 0.0225936	total: 343ms	remaining: 707ms
177:	learn: 0.0225123	total: 345ms	remaining: 705ms
178:	learn: 0.0224038	total: 346ms	remaining: 702ms
179:	learn: 0.0222610	total: 348ms	remaining: 699ms
180:	learn: 0.0219157	total: 349ms	remaining: 697ms
181:	learn: 0.0218471	total: 351ms	remaining: 694ms
182:	learn: 0.0216515	total: 353ms	remaining: 692ms
183:	learn: 0.0214675	total: 355ms	remaining: 690ms
184:	learn: 0.0212499	total: 357ms	remaining: 688ms
185:	learn: 0.0211961	total: 359ms	remaining: 686ms
186:	learn: 0.0210535	total: 361ms	remaining: 685ms
187:	learn: 0.0209919	total: 363ms	remaining: 683ms
188:	learn: 0.0208280	total: 365ms	remaining: 681ms
189:	learn: 0.0206659	total: 366ms	remaining: 679ms
190:	learn: 0.0205208	total: 368ms	remaining: 677ms
191:	learn: 0.0203860	total: 370ms	remaining: 675ms
192:	learn: 0.0203039	total: 372ms	remaining: 673ms
193:	learn: 0.0202858	total: 374ms	remaining: 670ms
194:	learn: 0.0201595	total: 375ms	remaining: 668ms
195:	learn: 0.0201449	total: 377ms	remaining: 665ms
196:	learn: 0.0200378	total: 379ms	remaining: 663ms
197:	learn: 0.0198771	total: 380ms	remaining: 661ms
198:	learn: 0.0197627	total: 382ms	remaining: 658ms
199:	learn: 0.0194590	total: 384ms	remaining: 656ms
200:	learn: 0.0194039	total: 386ms	remaining: 655ms
201:	learn: 0.0192109	total: 388ms	remaining: 653ms
202:	learn: 0.0191908	total: 390ms	remaining: 651ms
203:	learn: 0.0190590	total: 392ms	remaining: 649ms
204:	learn: 0.0189997	total: 394ms	remaining: 647ms
205:	learn: 0.0189293	total: 396ms	remaining: 646ms
206:	learn: 0.0187315	total: 398ms	remaining: 643ms
207:	learn: 0.0184743	total: 400ms	remaining: 642ms
208:	learn: 0.0182203	total: 402ms	remaining: 640ms
209:	learn: 0.0179932	total: 403ms	remaining: 638ms
210:	learn: 0.0178070	total: 405ms	remaining: 636ms
211:	learn: 0.0176843	total: 407ms	remaining: 634ms
212:	learn: 0.0175219	total: 409ms	remaining: 632ms
213:	learn: 0.0173931	total: 411ms	remaining: 629ms
214:	learn: 0.0172574	total: 413ms	remaining: 628ms
215:	learn: 0.0172431	total: 415ms	remaining: 627ms
216:	learn: 0.0171806	total: 417ms	remaining: 625ms
217:	learn: 0.0170522	total: 419ms	remaining: 623ms
218:	learn: 0.0168832	total: 421ms	remaining: 621ms
219:	learn: 0.0166707	total: 423ms	remaining: 619ms
220:	learn: 0.0166102	total: 425ms	remaining: 617ms
221:	learn: 0.0165744	total: 427ms	remaining: 615ms
222:	learn: 0.0164398	total: 429ms	remaining: 613ms
223:	learn: 0.0163709	total: 430ms	remaining: 611ms
224:	learn: 0.0162745	total: 432ms	remaining: 609ms
225:	learn: 0.0162122	total: 434ms	remaining: 607ms
226:	learn: 0.0160670	total: 436ms	remaining: 605ms
227:	learn: 0.0159791	total: 438ms	remaining: 603ms
228:	learn: 0.0159450	total: 439ms	remaining: 601ms
229:	learn: 0.0158943	total: 441ms	remaining: 599ms
230:	learn: 0.0158427	total: 443ms	remaining: 596ms
231:	learn: 0.0157636	total: 445ms	remaining: 594ms
232:	learn: 0.0157569	total: 446ms	remaining: 592ms
233:	learn: 0.0157278	total: 448ms	remaining: 590ms
234:	learn: 0.0156873	total: 450ms	remaining: 588ms
235:	learn: 0.0155970	total: 452ms	remaining: 586ms
236:	learn: 0.0154475	total: 454ms	remaining: 584ms
237:	learn: 0.0153116	total: 456ms	remaining: 582ms
238:	learn: 0.0153050	total: 458ms	remaining: 580ms
239:	learn: 0.0152128	total: 460ms	remaining: 578ms
240:	learn: 0.0151816	total: 462ms	remaining: 577ms
241:	learn: 0.0151435	total: 464ms	remaining: 575ms
242:	learn: 0.0150979	total: 465ms	remaining: 573ms
243:	learn: 0.0150181	total: 467ms	remaining: 571ms
244:	learn: 0.0149476	total: 469ms	remaining: 569ms
245:	learn: 0.0148299	total: 471ms	remaining: 567ms
246:	learn: 0.0147112	total: 472ms	remaining: 564ms
247:	learn: 0.0146722	total: 474ms	remaining: 562ms
248:	learn: 0.0145671	total: 476ms	remaining: 560ms
249:	learn: 0.0145356	total: 477ms	remaining: 558ms
250:	learn: 0.0144952	total: 479ms	remaining: 555ms
251:	learn: 0.0144280	total: 481ms	remaining: 553ms
252:	learn: 0.0144220	total: 482ms	remaining: 551ms
253:	learn: 0.0143516	total: 484ms	remaining: 549ms
254:	learn: 0.0143276	total: 485ms	remaining: 546ms
255:	learn: 0.0143218	total: 487ms	remaining: 544ms
256:	learn: 0.0142998	total: 489ms	remaining: 542ms
257:	learn: 0.0142363	total: 491ms	remaining: 540ms
258:	learn: 0.0141053	total: 493ms	remaining: 538ms
259:	learn: 0.0140229	total: 494ms	remaining: 536ms
260:	learn: 0.0139944	total: 496ms	remaining: 534ms
261:	learn: 0.0139752	total: 498ms	remaining: 532ms
262:	learn: 0.0138448	total: 500ms	remaining: 531ms
263:	learn: 0.0137367	total: 502ms	remaining: 529ms
264:	learn: 0.0136930	total: 504ms	remaining: 527ms
265:	learn: 0.0136342	total: 506ms	remaining: 525ms
266:	learn: 0.0135335	total: 507ms	remaining: 523ms
267:	learn: 0.0134973	total: 509ms	remaining: 520ms
268:	learn: 0.0134363	total: 511ms	remaining: 518ms
269:	learn: 0.0133699	total: 512ms	remaining: 516ms
270:	learn: 0.0133064	total: 514ms	remaining: 514ms
271:	learn: 0.0132725	total: 516ms	remaining: 512ms
272:	learn: 0.0132363	total: 518ms	remaining: 510ms
273:	learn: 0.0132144	total: 520ms	remaining: 508ms
274:	learn: 0.0131572	total: 521ms	remaining: 506ms
275:	learn: 0.0130568	total: 523ms	remaining: 504ms
276:	learn: 0.0129956	total: 525ms	remaining: 502ms
277:	learn: 0.0129168	total: 527ms	remaining: 500ms
278:	learn: 0.0128428	total: 528ms	remaining: 498ms
279:	learn: 0.0128093	total: 530ms	remaining: 496ms
280:	learn: 0.0127850	total: 532ms	remaining: 494ms
281:	learn: 0.0127605	total: 533ms	remaining: 492ms
282:	learn: 0.0127084	total: 535ms	remaining: 490ms
283:	learn: 0.0126534	total: 537ms	remaining: 488ms
284:	learn: 0.0126110	total: 538ms	remaining: 485ms
285:	learn: 0.0125626	total: 540ms	remaining: 483ms
286:	learn: 0.0125429	total: 541ms	remaining: 481ms
287:	learn: 0.0125194	total: 543ms	remaining: 479ms
288:	learn: 0.0124269	total: 544ms	remaining: 477ms
289:	learn: 0.0123495	total: 546ms	remaining: 475ms
290:	learn: 0.0123251	total: 548ms	remaining: 473ms
291:	learn: 0.0122594	total: 550ms	remaining: 471ms
292:	learn: 0.0122194	total: 552ms	remaining: 469ms
293:	learn: 0.0122043	total: 553ms	remaining: 467ms
294:	learn: 0.0121819	total: 555ms	remaining: 465ms
295:	learn: 0.0120861	total: 556ms	remaining: 462ms
296:	learn: 0.0120861	total: 558ms	remaining: 460ms
297:	learn: 0.0120236	total: 559ms	remaining: 458ms
298:	learn: 0.0119445	total: 561ms	remaining: 456ms
299:	learn: 0.0118685	total: 562ms	remaining: 454ms
300:	learn: 0.0118045	total: 564ms	remaining: 451ms
301:	learn: 0.0117321	total: 565ms	remaining: 449ms
302:	learn: 0.0116434	total: 567ms	remaining: 447ms
303:	learn: 0.0116035	total: 568ms	remaining: 445ms
304:	learn: 0.0115033	total: 570ms	remaining: 443ms
305:	learn: 0.0114111	total: 572ms	remaining: 441ms
306:	learn: 0.0113463	total: 573ms	remaining: 439ms
307:	learn: 0.0113191	total: 575ms	remaining: 437ms
308:	learn: 0.0112609	total: 576ms	remaining: 435ms
309:	learn: 0.0112383	total: 578ms	remaining: 433ms
310:	learn: 0.0112203	total: 580ms	remaining: 431ms
311:	learn: 0.0112009	total: 582ms	remaining: 429ms
312:	learn: 0.0111352	total: 585ms	remaining: 428ms
313:	learn: 0.0111151	total: 587ms	remaining: 426ms
314:	learn: 0.0110731	total: 588ms	remaining: 424ms
315:	learn: 0.0110718	total: 590ms	remaining: 422ms
316:	learn: 0.0110328	total: 592ms	remaining: 420ms
317:	learn: 0.0109714	total: 593ms	remaining: 418ms
318:	learn: 0.0108896	total: 595ms	remaining: 416ms
319:	learn: 0.0108750	total: 597ms	remaining: 414ms
320:	learn: 0.0108602	total: 598ms	remaining: 412ms
321:	learn: 0.0107992	total: 600ms	remaining: 410ms
322:	learn: 0.0107387	total: 602ms	remaining: 408ms
323:	learn: 0.0106866	total: 604ms	remaining: 406ms
324:	learn: 0.0106457	total: 606ms	remaining: 404ms
325:	learn: 0.0106098	total: 607ms	remaining: 402ms
326:	learn: 0.0105955	total: 609ms	remaining: 400ms
327:	learn: 0.0105383	total: 611ms	remaining: 398ms
328:	learn: 0.0104760	total: 612ms	remaining: 397ms
329:	learn: 0.0104542	total: 614ms	remaining: 395ms
330:	learn: 0.0104278	total: 616ms	remaining: 393ms
331:	learn: 0.0104278	total: 618ms	remaining: 391ms
332:	learn: 0.0103832	total: 619ms	remaining: 389ms
333:	learn: 0.0103417	total: 621ms	remaining: 387ms
334:	learn: 0.0103417	total: 623ms	remaining: 385ms
335:	learn: 0.0102816	total: 624ms	remaining: 383ms
336:	learn: 0.0102669	total: 626ms	remaining: 381ms
337:	learn: 0.0102250	total: 628ms	remaining: 379ms
338:	learn: 0.0101857	total: 630ms	remaining: 377ms
339:	learn: 0.0101518	total: 632ms	remaining: 375ms
340:	learn: 0.0100831	total: 633ms	remaining: 373ms
341:	learn: 0.0100706	total: 635ms	remaining: 372ms
342:	learn: 0.0100382	total: 637ms	remaining: 370ms
343:	learn: 0.0099858	total: 639ms	remaining: 368ms
344:	learn: 0.0099215	total: 641ms	remaining: 366ms
345:	learn: 0.0098915	total: 643ms	remaining: 364ms
346:	learn: 0.0098915	total: 645ms	remaining: 362ms
347:	learn: 0.0098533	total: 647ms	remaining: 361ms
348:	learn: 0.0098098	total: 648ms	remaining: 359ms
349:	learn: 0.0097490	total: 650ms	remaining: 356ms
350:	learn: 0.0096889	total: 651ms	remaining: 354ms
351:	learn: 0.0096561	total: 653ms	remaining: 352ms
352:	learn: 0.0096234	total: 654ms	remaining: 350ms
353:	learn: 0.0095669	total: 656ms	remaining: 348ms
354:	learn: 0.0095140	total: 657ms	remaining: 346ms
355:	learn: 0.0094944	total: 659ms	remaining: 344ms
356:	learn: 0.0094820	total: 661ms	remaining: 342ms
357:	learn: 0.0094406	total: 662ms	remaining: 340ms
358:	learn: 0.0094296	total: 664ms	remaining: 338ms
359:	learn: 0.0093818	total: 665ms	remaining: 336ms
360:	learn: 0.0093402	total: 667ms	remaining: 335ms
361:	learn: 0.0092944	total: 669ms	remaining: 333ms
362:	learn: 0.0092564	total: 671ms	remaining: 331ms
363:	learn: 0.0092115	total: 673ms	remaining: 329ms
364:	learn: 0.0091711	total: 674ms	remaining: 327ms
365:	learn: 0.0091285	total: 676ms	remaining: 325ms
366:	learn: 0.0091284	total: 678ms	remaining: 323ms
367:	learn: 0.0091212	total: 680ms	remaining: 321ms
368:	learn: 0.0091095	total: 682ms	remaining: 320ms
369:	learn: 0.0090988	total: 684ms	remaining: 318ms
370:	learn: 0.0090438	total: 685ms	remaining: 316ms
371:	learn: 0.0090123	total: 687ms	remaining: 314ms
372:	learn: 0.0090028	total: 689ms	remaining: 312ms
373:	learn: 0.0089776	total: 690ms	remaining: 310ms
374:	learn: 0.0089395	total: 692ms	remaining: 308ms
375:	learn: 0.0089004	total: 693ms	remaining: 306ms
376:	learn: 0.0088754	total: 695ms	remaining: 304ms
377:	learn: 0.0088345	total: 696ms	remaining: 302ms
378:	learn: 0.0088145	total: 698ms	remaining: 300ms
379:	learn: 0.0087760	total: 699ms	remaining: 298ms
380:	learn: 0.0087760	total: 701ms	remaining: 296ms
381:	learn: 0.0087378	total: 703ms	remaining: 294ms
382:	learn: 0.0087287	total: 704ms	remaining: 292ms
383:	learn: 0.0087015	total: 706ms	remaining: 290ms
384:	learn: 0.0086595	total: 707ms	remaining: 288ms
385:	learn: 0.0086107	total: 709ms	remaining: 286ms
386:	learn: 0.0085733	total: 710ms	remaining: 284ms
387:	learn: 0.0085412	total: 712ms	remaining: 283ms
388:	learn: 0.0085091	total: 713ms	remaining: 281ms
389:	learn: 0.0084839	total: 715ms	remaining: 279ms
390:	learn: 0.0084443	total: 717ms	remaining: 277ms
391:	learn: 0.0084239	total: 718ms	remaining: 275ms
392:	learn: 0.0083891	total: 720ms	remaining: 273ms
393:	learn: 0.0083768	total: 721ms	remaining: 271ms
394:	learn: 0.0083230	total: 723ms	remaining: 269ms
395:	learn: 0.0083010	total: 725ms	remaining: 267ms
396:	learn: 0.0082596	total: 726ms	remaining: 265ms
397:	learn: 0.0082273	total: 728ms	remaining: 263ms
398:	learn: 0.0082056	total: 730ms	remaining: 262ms
399:	learn: 0.0081759	total: 732ms	remaining: 260ms
400:	learn: 0.0081360	total: 734ms	remaining: 258ms
401:	learn: 0.0081021	total: 735ms	remaining: 256ms
402:	learn: 0.0080854	total: 737ms	remaining: 254ms
403:	learn: 0.0080544	total: 739ms	remaining: 252ms
404:	learn: 0.0080222	total: 740ms	remaining: 250ms
405:	learn: 0.0080021	total: 742ms	remaining: 249ms
406:	learn: 0.0079692	total: 744ms	remaining: 247ms
407:	learn: 0.0079474	total: 746ms	remaining: 245ms
408:	learn: 0.0079401	total: 747ms	remaining: 243ms
409:	learn: 0.0078990	total: 749ms	remaining: 241ms
410:	learn: 0.0078910	total: 750ms	remaining: 239ms
411:	learn: 0.0078525	total: 752ms	remaining: 237ms
412:	learn: 0.0078174	total: 753ms	remaining: 235ms
413:	learn: 0.0077868	total: 755ms	remaining: 233ms
414:	learn: 0.0077698	total: 756ms	remaining: 231ms
415:	learn: 0.0077539	total: 758ms	remaining: 230ms
416:	learn: 0.0077261	total: 760ms	remaining: 228ms
417:	learn: 0.0077068	total: 762ms	remaining: 226ms
418:	learn: 0.0076832	total: 764ms	remaining: 224ms
419:	learn: 0.0076522	total: 766ms	remaining: 223ms
420:	learn: 0.0076354	total: 768ms	remaining: 221ms
421:	learn: 0.0076167	total: 770ms	remaining: 219ms
422:	learn: 0.0075854	total: 771ms	remaining: 217ms
423:	learn: 0.0075637	total: 773ms	remaining: 215ms
424:	learn: 0.0075541	total: 775ms	remaining: 213ms
425:	learn: 0.0075282	total: 777ms	remaining: 212ms
426:	learn: 0.0075035	total: 779ms	remaining: 210ms
427:	learn: 0.0074862	total: 780ms	remaining: 208ms
428:	learn: 0.0074613	total: 782ms	remaining: 206ms
429:	learn: 0.0074337	total: 783ms	remaining: 204ms
430:	learn: 0.0074269	total: 785ms	remaining: 202ms
431:	learn: 0.0074055	total: 786ms	remaining: 200ms
432:	learn: 0.0073919	total: 788ms	remaining: 198ms
433:	learn: 0.0073732	total: 790ms	remaining: 197ms
434:	learn: 0.0073472	total: 792ms	remaining: 195ms
435:	learn: 0.0073469	total: 793ms	remaining: 193ms
436:	learn: 0.0073234	total: 795ms	remaining: 191ms
437:	learn: 0.0073234	total: 796ms	remaining: 189ms
438:	learn: 0.0072982	total: 798ms	remaining: 187ms
439:	learn: 0.0072537	total: 799ms	remaining: 185ms
440:	learn: 0.0072345	total: 801ms	remaining: 183ms
441:	learn: 0.0072294	total: 802ms	remaining: 182ms
442:	learn: 0.0072167	total: 804ms	remaining: 180ms
443:	learn: 0.0071978	total: 806ms	remaining: 178ms
444:	learn: 0.0071679	total: 807ms	remaining: 176ms
445:	learn: 0.0071514	total: 809ms	remaining: 174ms
446:	learn: 0.0071285	total: 811ms	remaining: 172ms
447:	learn: 0.0071104	total: 812ms	remaining: 170ms
448:	learn: 0.0071104	total: 813ms	remaining: 168ms
449:	learn: 0.0070849	total: 815ms	remaining: 167ms
450:	learn: 0.0070680	total: 816ms	remaining: 165ms
451:	learn: 0.0070424	total: 818ms	remaining: 163ms
452:	learn: 0.0070212	total: 820ms	remaining: 161ms
453:	learn: 0.0069999	total: 821ms	remaining: 159ms
454:	learn: 0.0069714	total: 823ms	remaining: 157ms
455:	learn: 0.0069508	total: 824ms	remaining: 155ms
456:	learn: 0.0069392	total: 826ms	remaining: 154ms
457:	learn: 0.0069059	total: 827ms	remaining: 152ms
458:	learn: 0.0068872	total: 829ms	remaining: 150ms
459:	learn: 0.0068704	total: 830ms	remaining: 148ms
460:	learn: 0.0068499	total: 832ms	remaining: 146ms
461:	learn: 0.0068350	total: 834ms	remaining: 144ms
462:	learn: 0.0068299	total: 835ms	remaining: 143ms
463:	learn: 0.0068123	total: 837ms	remaining: 141ms
464:	learn: 0.0068123	total: 839ms	remaining: 139ms
465:	learn: 0.0067873	total: 841ms	remaining: 137ms
466:	learn: 0.0067822	total: 842ms	remaining: 135ms
467:	learn: 0.0067683	total: 844ms	remaining: 133ms
468:	learn: 0.0067425	total: 846ms	remaining: 132ms
469:	learn: 0.0067196	total: 847ms	remaining: 130ms
470:	learn: 0.0067143	total: 849ms	remaining: 128ms
471:	learn: 0.0067014	total: 851ms	remaining: 126ms
472:	learn: 0.0066790	total: 852ms	remaining: 124ms
473:	learn: 0.0066507	total: 854ms	remaining: 123ms
474:	learn: 0.0066399	total: 856ms	remaining: 121ms
475:	learn: 0.0066215	total: 858ms	remaining: 119ms
476:	learn: 0.0066043	total: 859ms	remaining: 117ms
477:	learn: 0.0066040	total: 860ms	remaining: 115ms
478:	learn: 0.0065863	total: 862ms	remaining: 113ms
479:	learn: 0.0065760	total: 864ms	remaining: 112ms
480:	learn: 0.0065703	total: 865ms	remaining: 110ms
481:	learn: 0.0065609	total: 867ms	remaining: 108ms
482:	learn: 0.0065440	total: 869ms	remaining: 106ms
483:	learn: 0.0065275	total: 871ms	remaining: 104ms
484:	learn: 0.0065194	total: 872ms	remaining: 103ms
485:	learn: 0.0065074	total: 874ms	remaining: 101ms
486:	learn: 0.0064941	total: 876ms	remaining: 98.9ms
487:	learn: 0.0064915	total: 877ms	remaining: 97.1ms
488:	learn: 0.0064752	total: 879ms	remaining: 95.3ms
489:	learn: 0.0064678	total: 881ms	remaining: 93.4ms
490:	learn: 0.0064524	total: 882ms	remaining: 91.6ms
491:	learn: 0.0064508	total: 884ms	remaining: 89.8ms
492:	learn: 0.0064364	total: 886ms	remaining: 88ms
493:	learn: 0.0064298	total: 887ms	remaining: 86.2ms
494:	learn: 0.0064042	total: 889ms	remaining: 84.4ms
495:	learn: 0.0063809	total: 890ms	remaining: 82.6ms
496:	learn: 0.0063665	total: 892ms	remaining: 80.8ms
497:	learn: 0.0063664	total: 894ms	remaining: 78.9ms
498:	learn: 0.0063487	total: 895ms	remaining: 77.2ms
499:	learn: 0.0063215	total: 897ms	remaining: 75.4ms
500:	learn: 0.0063141	total: 899ms	remaining: 73.6ms
501:	learn: 0.0063138	total: 900ms	remaining: 71.7ms
502:	learn: 0.0062912	total: 902ms	remaining: 69.9ms
503:	learn: 0.0062910	total: 903ms	remaining: 68.1ms
504:	learn: 0.0062728	total: 905ms	remaining: 66.3ms
505:	learn: 0.0062727	total: 907ms	remaining: 64.5ms
506:	learn: 0.0062517	total: 908ms	remaining: 62.7ms
507:	learn: 0.0062356	total: 910ms	remaining: 60.9ms
508:	learn: 0.0062082	total: 912ms	remaining: 59.1ms
509:	learn: 0.0061948	total: 913ms	remaining: 57.3ms
510:	learn: 0.0061612	total: 915ms	remaining: 55.5ms
511:	learn: 0.0061360	total: 916ms	remaining: 53.7ms
512:	learn: 0.0061217	total: 918ms	remaining: 51.9ms
513:	learn: 0.0061014	total: 919ms	remaining: 50.1ms
514:	learn: 0.0060844	total: 921ms	remaining: 48.3ms
515:	learn: 0.0060781	total: 922ms	remaining: 46.5ms
516:	learn: 0.0060641	total: 924ms	remaining: 44.7ms
517:	learn: 0.0060628	total: 926ms	remaining: 42.9ms
518:	learn: 0.0060393	total: 928ms	remaining: 41.1ms
519:	learn: 0.0060353	total: 929ms	remaining: 39.3ms
520:	learn: 0.0060316	total: 931ms	remaining: 37.5ms
521:	learn: 0.0060316	total: 932ms	remaining: 35.7ms
522:	learn: 0.0060165	total: 934ms	remaining: 33.9ms
523:	learn: 0.0060104	total: 935ms	remaining: 32.1ms
524:	learn: 0.0060033	total: 937ms	remaining: 30.3ms
525:	learn: 0.0059837	total: 938ms	remaining: 28.5ms
526:	learn: 0.0059682	total: 940ms	remaining: 26.8ms
527:	learn: 0.0059537	total: 941ms	remaining: 25ms
528:	learn: 0.0059435	total: 943ms	remaining: 23.2ms
529:	learn: 0.0059203	total: 945ms	remaining: 21.4ms
530:	learn: 0.0059149	total: 946ms	remaining: 19.6ms
531:	learn: 0.0059101	total: 948ms	remaining: 17.8ms
532:	learn: 0.0059046	total: 949ms	remaining: 16ms
533:	learn: 0.0058867	total: 951ms	remaining: 14.2ms
534:	learn: 0.0058813	total: 952ms	remaining: 12.5ms
535:	learn: 0.0058624	total: 954ms	remaining: 10.7ms
536:	learn: 0.0058439	total: 956ms	remaining: 8.9ms
537:	learn: 0.0058438	total: 958ms	remaining: 7.12ms
538:	learn: 0.0058387	total: 959ms	remaining: 5.34ms
539:	learn: 0.0058334	total: 961ms	remaining: 3.56ms
540:	learn: 0.0058191	total: 963ms	remaining: 1.78ms
541:	learn: 0.0058044	total: 965ms	remaining: 0us
0:	learn: 0.5760504	total: 1.76ms	remaining: 953ms
1:	learn: 0.4713877	total: 3.63ms	remaining: 980ms
2:	learn: 0.4031854	total: 5.25ms	remaining: 944ms
3:	learn: 0.3553157	total: 7.01ms	remaining: 944ms
4:	learn: 0.3298034	total: 8.75ms	remaining: 940ms
5:	learn: 0.3094960	total: 10.6ms	remaining: 945ms
6:	learn: 0.2941595	total: 12.4ms	remaining: 951ms
7:	learn: 0.2772337	total: 14.3ms	remaining: 951ms
8:	learn: 0.2628721	total: 15.8ms	remaining: 938ms
9:	learn: 0.2574092	total: 17.1ms	remaining: 911ms
10:	learn: 0.2448794	total: 18.8ms	remaining: 906ms
11:	learn: 0.2419418	total: 20.3ms	remaining: 898ms
12:	learn: 0.2327553	total: 22.1ms	remaining: 901ms
13:	learn: 0.2249596	total: 23.7ms	remaining: 896ms
14:	learn: 0.2205127	total: 25.4ms	remaining: 893ms
15:	learn: 0.2147860	total: 27.1ms	remaining: 891ms
16:	learn: 0.2072371	total: 28.6ms	remaining: 884ms
17:	learn: 0.2026455	total: 30.3ms	remaining: 881ms
18:	learn: 0.1960631	total: 32ms	remaining: 880ms
19:	learn: 0.1892579	total: 33.6ms	remaining: 877ms
20:	learn: 0.1862977	total: 35.4ms	remaining: 878ms
21:	learn: 0.1797642	total: 37ms	remaining: 875ms
22:	learn: 0.1723097	total: 38.6ms	remaining: 872ms
23:	learn: 0.1670428	total: 40.4ms	remaining: 872ms
24:	learn: 0.1609536	total: 42.6ms	remaining: 880ms
25:	learn: 0.1563956	total: 44.4ms	remaining: 880ms
26:	learn: 0.1517267	total: 46.2ms	remaining: 881ms
27:	learn: 0.1496505	total: 47.9ms	remaining: 880ms
28:	learn: 0.1461032	total: 49.6ms	remaining: 878ms
29:	learn: 0.1415452	total: 51.2ms	remaining: 873ms
30:	learn: 0.1380990	total: 52.8ms	remaining: 871ms
31:	learn: 0.1337286	total: 54.7ms	remaining: 872ms
32:	learn: 0.1284169	total: 56.5ms	remaining: 871ms
33:	learn: 0.1254221	total: 58.4ms	remaining: 872ms
34:	learn: 0.1228747	total: 60.3ms	remaining: 873ms
35:	learn: 0.1212267	total: 61.8ms	remaining: 869ms
36:	learn: 0.1173931	total: 63.3ms	remaining: 864ms
37:	learn: 0.1148499	total: 64.9ms	remaining: 861ms
38:	learn: 0.1129666	total: 66.6ms	remaining: 860ms
39:	learn: 0.1123077	total: 68.2ms	remaining: 856ms
40:	learn: 0.1103712	total: 70ms	remaining: 855ms
41:	learn: 0.1083265	total: 71.8ms	remaining: 855ms
42:	learn: 0.1064435	total: 73.6ms	remaining: 854ms
43:	learn: 0.1042188	total: 75.3ms	remaining: 853ms
44:	learn: 0.1004505	total: 77ms	remaining: 850ms
45:	learn: 0.0964920	total: 78.5ms	remaining: 846ms
46:	learn: 0.0946511	total: 80ms	remaining: 842ms
47:	learn: 0.0927353	total: 81.7ms	remaining: 841ms
48:	learn: 0.0922038	total: 83.3ms	remaining: 838ms
49:	learn: 0.0898481	total: 85ms	remaining: 836ms
50:	learn: 0.0891110	total: 86.6ms	remaining: 834ms
51:	learn: 0.0868332	total: 88.2ms	remaining: 831ms
52:	learn: 0.0853167	total: 89.8ms	remaining: 828ms
53:	learn: 0.0829132	total: 91.4ms	remaining: 826ms
54:	learn: 0.0800560	total: 93ms	remaining: 824ms
55:	learn: 0.0778902	total: 94.7ms	remaining: 822ms
56:	learn: 0.0768472	total: 96.5ms	remaining: 821ms
57:	learn: 0.0753410	total: 98.1ms	remaining: 819ms
58:	learn: 0.0739420	total: 99.7ms	remaining: 816ms
59:	learn: 0.0737478	total: 101ms	remaining: 814ms
60:	learn: 0.0724914	total: 103ms	remaining: 813ms
61:	learn: 0.0706403	total: 105ms	remaining: 811ms
62:	learn: 0.0702050	total: 106ms	remaining: 807ms
63:	learn: 0.0700183	total: 108ms	remaining: 804ms
64:	learn: 0.0696063	total: 109ms	remaining: 802ms
65:	learn: 0.0682672	total: 111ms	remaining: 799ms
66:	learn: 0.0674068	total: 112ms	remaining: 796ms
67:	learn: 0.0665534	total: 114ms	remaining: 793ms
68:	learn: 0.0648671	total: 115ms	remaining: 790ms
69:	learn: 0.0635656	total: 117ms	remaining: 787ms
70:	learn: 0.0635009	total: 118ms	remaining: 784ms
71:	learn: 0.0621819	total: 120ms	remaining: 781ms
72:	learn: 0.0620421	total: 121ms	remaining: 779ms
73:	learn: 0.0617751	total: 123ms	remaining: 776ms
74:	learn: 0.0616279	total: 124ms	remaining: 773ms
75:	learn: 0.0614121	total: 126ms	remaining: 770ms
76:	learn: 0.0600040	total: 127ms	remaining: 768ms
77:	learn: 0.0597502	total: 129ms	remaining: 768ms
78:	learn: 0.0588887	total: 131ms	remaining: 767ms
79:	learn: 0.0581408	total: 133ms	remaining: 767ms
80:	learn: 0.0567778	total: 135ms	remaining: 766ms
81:	learn: 0.0558820	total: 136ms	remaining: 764ms
82:	learn: 0.0556477	total: 138ms	remaining: 763ms
83:	learn: 0.0554455	total: 164ms	remaining: 896ms
84:	learn: 0.0554145	total: 166ms	remaining: 891ms
85:	learn: 0.0544598	total: 167ms	remaining: 887ms
86:	learn: 0.0542166	total: 169ms	remaining: 883ms
87:	learn: 0.0540256	total: 170ms	remaining: 878ms
88:	learn: 0.0535770	total: 172ms	remaining: 874ms
89:	learn: 0.0527645	total: 173ms	remaining: 871ms
90:	learn: 0.0522048	total: 175ms	remaining: 867ms
91:	learn: 0.0508780	total: 176ms	remaining: 863ms
92:	learn: 0.0500671	total: 178ms	remaining: 859ms
93:	learn: 0.0491523	total: 179ms	remaining: 855ms
94:	learn: 0.0489937	total: 181ms	remaining: 852ms
95:	learn: 0.0488510	total: 182ms	remaining: 847ms
96:	learn: 0.0478261	total: 184ms	remaining: 843ms
97:	learn: 0.0473989	total: 185ms	remaining: 840ms
98:	learn: 0.0462650	total: 187ms	remaining: 836ms
99:	learn: 0.0462199	total: 188ms	remaining: 833ms
100:	learn: 0.0460288	total: 190ms	remaining: 829ms
101:	learn: 0.0452249	total: 191ms	remaining: 826ms
102:	learn: 0.0443822	total: 194ms	remaining: 825ms
103:	learn: 0.0438090	total: 195ms	remaining: 823ms
104:	learn: 0.0437300	total: 198ms	remaining: 822ms
105:	learn: 0.0435779	total: 199ms	remaining: 819ms
106:	learn: 0.0428164	total: 201ms	remaining: 818ms
107:	learn: 0.0422963	total: 203ms	remaining: 816ms
108:	learn: 0.0416940	total: 205ms	remaining: 815ms
109:	learn: 0.0415363	total: 207ms	remaining: 813ms
110:	learn: 0.0412214	total: 209ms	remaining: 812ms
111:	learn: 0.0411125	total: 211ms	remaining: 809ms
112:	learn: 0.0409731	total: 212ms	remaining: 806ms
113:	learn: 0.0409237	total: 214ms	remaining: 804ms
114:	learn: 0.0403814	total: 216ms	remaining: 801ms
115:	learn: 0.0398177	total: 217ms	remaining: 798ms
116:	learn: 0.0392151	total: 219ms	remaining: 795ms
117:	learn: 0.0385481	total: 221ms	remaining: 793ms
118:	learn: 0.0378566	total: 222ms	remaining: 790ms
119:	learn: 0.0376535	total: 224ms	remaining: 789ms
120:	learn: 0.0371884	total: 226ms	remaining: 787ms
121:	learn: 0.0368770	total: 228ms	remaining: 785ms
122:	learn: 0.0363002	total: 230ms	remaining: 782ms
123:	learn: 0.0359771	total: 231ms	remaining: 780ms
124:	learn: 0.0355494	total: 233ms	remaining: 778ms
125:	learn: 0.0350427	total: 235ms	remaining: 777ms
126:	learn: 0.0346290	total: 267ms	remaining: 873ms
127:	learn: 0.0341290	total: 269ms	remaining: 870ms
128:	learn: 0.0337906	total: 270ms	remaining: 866ms
129:	learn: 0.0332653	total: 272ms	remaining: 862ms
130:	learn: 0.0331900	total: 274ms	remaining: 858ms
131:	learn: 0.0327707	total: 275ms	remaining: 855ms
132:	learn: 0.0324096	total: 277ms	remaining: 851ms
133:	learn: 0.0320956	total: 278ms	remaining: 847ms
134:	learn: 0.0320269	total: 280ms	remaining: 844ms
135:	learn: 0.0319707	total: 282ms	remaining: 841ms
136:	learn: 0.0316969	total: 283ms	remaining: 837ms
137:	learn: 0.0316137	total: 285ms	remaining: 834ms
138:	learn: 0.0313662	total: 286ms	remaining: 830ms
139:	learn: 0.0306796	total: 288ms	remaining: 826ms
140:	learn: 0.0302570	total: 289ms	remaining: 822ms
141:	learn: 0.0300469	total: 291ms	remaining: 818ms
142:	learn: 0.0295623	total: 292ms	remaining: 815ms
143:	learn: 0.0292377	total: 294ms	remaining: 811ms
144:	learn: 0.0290300	total: 295ms	remaining: 808ms
145:	learn: 0.0287753	total: 297ms	remaining: 806ms
146:	learn: 0.0282587	total: 299ms	remaining: 803ms
147:	learn: 0.0280179	total: 301ms	remaining: 801ms
148:	learn: 0.0277695	total: 303ms	remaining: 798ms
149:	learn: 0.0276312	total: 304ms	remaining: 796ms
150:	learn: 0.0274551	total: 306ms	remaining: 792ms
151:	learn: 0.0274079	total: 308ms	remaining: 789ms
152:	learn: 0.0272539	total: 309ms	remaining: 787ms
153:	learn: 0.0269796	total: 311ms	remaining: 784ms
154:	learn: 0.0267719	total: 313ms	remaining: 781ms
155:	learn: 0.0267001	total: 315ms	remaining: 779ms
156:	learn: 0.0266334	total: 316ms	remaining: 776ms
157:	learn: 0.0263308	total: 318ms	remaining: 773ms
158:	learn: 0.0262010	total: 320ms	remaining: 770ms
159:	learn: 0.0261410	total: 321ms	remaining: 767ms
160:	learn: 0.0258417	total: 323ms	remaining: 764ms
161:	learn: 0.0256227	total: 324ms	remaining: 761ms
162:	learn: 0.0254378	total: 326ms	remaining: 757ms
163:	learn: 0.0252142	total: 327ms	remaining: 754ms
164:	learn: 0.0249072	total: 329ms	remaining: 751ms
165:	learn: 0.0246598	total: 331ms	remaining: 749ms
166:	learn: 0.0243290	total: 333ms	remaining: 748ms
167:	learn: 0.0242348	total: 335ms	remaining: 745ms
168:	learn: 0.0241851	total: 336ms	remaining: 743ms
169:	learn: 0.0241395	total: 338ms	remaining: 740ms
170:	learn: 0.0236888	total: 340ms	remaining: 738ms
171:	learn: 0.0234510	total: 342ms	remaining: 735ms
172:	learn: 0.0234127	total: 343ms	remaining: 732ms
173:	learn: 0.0233540	total: 345ms	remaining: 730ms
174:	learn: 0.0230286	total: 347ms	remaining: 728ms
175:	learn: 0.0230257	total: 349ms	remaining: 725ms
176:	learn: 0.0227587	total: 350ms	remaining: 722ms
177:	learn: 0.0227217	total: 352ms	remaining: 720ms
178:	learn: 0.0226835	total: 354ms	remaining: 717ms
179:	learn: 0.0226478	total: 355ms	remaining: 714ms
180:	learn: 0.0223114	total: 357ms	remaining: 712ms
181:	learn: 0.0219766	total: 359ms	remaining: 709ms
182:	learn: 0.0217573	total: 360ms	remaining: 707ms
183:	learn: 0.0216022	total: 362ms	remaining: 705ms
184:	learn: 0.0214122	total: 364ms	remaining: 703ms
185:	learn: 0.0212444	total: 366ms	remaining: 701ms
186:	learn: 0.0211535	total: 368ms	remaining: 698ms
187:	learn: 0.0209425	total: 369ms	remaining: 696ms
188:	learn: 0.0207644	total: 371ms	remaining: 694ms
189:	learn: 0.0206382	total: 373ms	remaining: 692ms
190:	learn: 0.0204955	total: 375ms	remaining: 690ms
191:	learn: 0.0204631	total: 377ms	remaining: 687ms
192:	learn: 0.0204582	total: 379ms	remaining: 685ms
193:	learn: 0.0204268	total: 380ms	remaining: 682ms
194:	learn: 0.0201896	total: 382ms	remaining: 680ms
195:	learn: 0.0200863	total: 384ms	remaining: 677ms
196:	learn: 0.0200572	total: 385ms	remaining: 675ms
197:	learn: 0.0199190	total: 387ms	remaining: 673ms
198:	learn: 0.0197450	total: 389ms	remaining: 671ms
199:	learn: 0.0196876	total: 392ms	remaining: 669ms
200:	learn: 0.0196592	total: 393ms	remaining: 667ms
201:	learn: 0.0195984	total: 395ms	remaining: 665ms
202:	learn: 0.0194019	total: 397ms	remaining: 662ms
203:	learn: 0.0193056	total: 399ms	remaining: 660ms
204:	learn: 0.0192469	total: 400ms	remaining: 658ms
205:	learn: 0.0192274	total: 402ms	remaining: 655ms
206:	learn: 0.0190149	total: 403ms	remaining: 652ms
207:	learn: 0.0189649	total: 404ms	remaining: 649ms
208:	learn: 0.0189390	total: 406ms	remaining: 647ms
209:	learn: 0.0188699	total: 408ms	remaining: 644ms
210:	learn: 0.0188156	total: 409ms	remaining: 642ms
211:	learn: 0.0185372	total: 411ms	remaining: 639ms
212:	learn: 0.0184046	total: 412ms	remaining: 637ms
213:	learn: 0.0183100	total: 414ms	remaining: 634ms
214:	learn: 0.0182862	total: 416ms	remaining: 632ms
215:	learn: 0.0182693	total: 417ms	remaining: 629ms
216:	learn: 0.0181887	total: 419ms	remaining: 627ms
217:	learn: 0.0179699	total: 420ms	remaining: 625ms
218:	learn: 0.0177719	total: 422ms	remaining: 622ms
219:	learn: 0.0176388	total: 424ms	remaining: 620ms
220:	learn: 0.0174648	total: 425ms	remaining: 618ms
221:	learn: 0.0173237	total: 427ms	remaining: 616ms
222:	learn: 0.0171527	total: 429ms	remaining: 613ms
223:	learn: 0.0170800	total: 431ms	remaining: 612ms
224:	learn: 0.0169395	total: 432ms	remaining: 609ms
225:	learn: 0.0168199	total: 434ms	remaining: 607ms
226:	learn: 0.0166587	total: 436ms	remaining: 605ms
227:	learn: 0.0165059	total: 438ms	remaining: 603ms
228:	learn: 0.0164143	total: 440ms	remaining: 602ms
229:	learn: 0.0162870	total: 442ms	remaining: 599ms
230:	learn: 0.0161440	total: 443ms	remaining: 597ms
231:	learn: 0.0160314	total: 445ms	remaining: 594ms
232:	learn: 0.0159143	total: 447ms	remaining: 592ms
233:	learn: 0.0158030	total: 448ms	remaining: 590ms
234:	learn: 0.0156758	total: 450ms	remaining: 587ms
235:	learn: 0.0155144	total: 451ms	remaining: 585ms
236:	learn: 0.0154055	total: 453ms	remaining: 583ms
237:	learn: 0.0152814	total: 455ms	remaining: 581ms
238:	learn: 0.0152122	total: 457ms	remaining: 579ms
239:	learn: 0.0151372	total: 458ms	remaining: 577ms
240:	learn: 0.0150202	total: 460ms	remaining: 574ms
241:	learn: 0.0149232	total: 462ms	remaining: 572ms
242:	learn: 0.0148735	total: 463ms	remaining: 570ms
243:	learn: 0.0147914	total: 465ms	remaining: 568ms
244:	learn: 0.0147350	total: 467ms	remaining: 566ms
245:	learn: 0.0146282	total: 468ms	remaining: 563ms
246:	learn: 0.0144846	total: 470ms	remaining: 561ms
247:	learn: 0.0144314	total: 472ms	remaining: 559ms
248:	learn: 0.0143898	total: 473ms	remaining: 557ms
249:	learn: 0.0143616	total: 475ms	remaining: 554ms
250:	learn: 0.0142707	total: 476ms	remaining: 552ms
251:	learn: 0.0142080	total: 478ms	remaining: 550ms
252:	learn: 0.0141047	total: 479ms	remaining: 548ms
253:	learn: 0.0140505	total: 481ms	remaining: 546ms
254:	learn: 0.0139484	total: 483ms	remaining: 544ms
255:	learn: 0.0139013	total: 485ms	remaining: 541ms
256:	learn: 0.0138073	total: 486ms	remaining: 539ms
257:	learn: 0.0137154	total: 488ms	remaining: 537ms
258:	learn: 0.0136137	total: 489ms	remaining: 535ms
259:	learn: 0.0136137	total: 500ms	remaining: 542ms
260:	learn: 0.0135111	total: 501ms	remaining: 539ms
261:	learn: 0.0134743	total: 503ms	remaining: 537ms
262:	learn: 0.0134159	total: 504ms	remaining: 535ms
263:	learn: 0.0133636	total: 506ms	remaining: 533ms
264:	learn: 0.0133031	total: 507ms	remaining: 530ms
265:	learn: 0.0132373	total: 509ms	remaining: 528ms
266:	learn: 0.0132371	total: 510ms	remaining: 526ms
267:	learn: 0.0131615	total: 512ms	remaining: 524ms
268:	learn: 0.0130980	total: 514ms	remaining: 522ms
269:	learn: 0.0129999	total: 516ms	remaining: 520ms
270:	learn: 0.0128982	total: 517ms	remaining: 517ms
271:	learn: 0.0128184	total: 519ms	remaining: 515ms
272:	learn: 0.0128184	total: 521ms	remaining: 513ms
273:	learn: 0.0128071	total: 522ms	remaining: 511ms
274:	learn: 0.0128069	total: 524ms	remaining: 508ms
275:	learn: 0.0128069	total: 525ms	remaining: 506ms
276:	learn: 0.0128068	total: 526ms	remaining: 504ms
277:	learn: 0.0128067	total: 528ms	remaining: 501ms
278:	learn: 0.0128066	total: 529ms	remaining: 499ms
279:	learn: 0.0128060	total: 531ms	remaining: 497ms
280:	learn: 0.0128058	total: 532ms	remaining: 494ms
281:	learn: 0.0128058	total: 533ms	remaining: 492ms
282:	learn: 0.0128058	total: 535ms	remaining: 489ms
283:	learn: 0.0128057	total: 536ms	remaining: 487ms
284:	learn: 0.0128056	total: 537ms	remaining: 485ms
285:	learn: 0.0128056	total: 539ms	remaining: 482ms
286:	learn: 0.0128057	total: 540ms	remaining: 480ms
287:	learn: 0.0128054	total: 542ms	remaining: 478ms
288:	learn: 0.0128054	total: 544ms	remaining: 476ms
289:	learn: 0.0128054	total: 545ms	remaining: 474ms
290:	learn: 0.0128054	total: 546ms	remaining: 471ms
291:	learn: 0.0128054	total: 548ms	remaining: 469ms
292:	learn: 0.0128054	total: 549ms	remaining: 467ms
293:	learn: 0.0128048	total: 551ms	remaining: 464ms
294:	learn: 0.0128047	total: 552ms	remaining: 462ms
295:	learn: 0.0128046	total: 553ms	remaining: 460ms
296:	learn: 0.0128046	total: 555ms	remaining: 458ms
297:	learn: 0.0128044	total: 556ms	remaining: 455ms
298:	learn: 0.0128043	total: 558ms	remaining: 453ms
299:	learn: 0.0128042	total: 559ms	remaining: 451ms
300:	learn: 0.0128038	total: 560ms	remaining: 449ms
301:	learn: 0.0128038	total: 562ms	remaining: 446ms
302:	learn: 0.0128038	total: 563ms	remaining: 444ms
303:	learn: 0.0128038	total: 565ms	remaining: 442ms
304:	learn: 0.0128037	total: 566ms	remaining: 440ms
305:	learn: 0.0128037	total: 567ms	remaining: 437ms
306:	learn: 0.0128036	total: 569ms	remaining: 435ms
307:	learn: 0.0128034	total: 570ms	remaining: 433ms
308:	learn: 0.0128030	total: 572ms	remaining: 431ms
309:	learn: 0.0128030	total: 573ms	remaining: 429ms
310:	learn: 0.0128028	total: 575ms	remaining: 427ms
311:	learn: 0.0128023	total: 576ms	remaining: 425ms
312:	learn: 0.0128022	total: 578ms	remaining: 423ms
313:	learn: 0.0128021	total: 579ms	remaining: 421ms
314:	learn: 0.0128019	total: 581ms	remaining: 419ms
315:	learn: 0.0128017	total: 582ms	remaining: 416ms
316:	learn: 0.0128017	total: 584ms	remaining: 414ms
317:	learn: 0.0128017	total: 585ms	remaining: 412ms
318:	learn: 0.0128017	total: 587ms	remaining: 410ms
319:	learn: 0.0128017	total: 588ms	remaining: 408ms
320:	learn: 0.0128016	total: 589ms	remaining: 406ms
321:	learn: 0.0128016	total: 591ms	remaining: 404ms
322:	learn: 0.0128015	total: 592ms	remaining: 401ms
323:	learn: 0.0128009	total: 593ms	remaining: 399ms
324:	learn: 0.0128008	total: 595ms	remaining: 397ms
325:	learn: 0.0128007	total: 596ms	remaining: 395ms
326:	learn: 0.0128007	total: 598ms	remaining: 393ms
327:	learn: 0.0128005	total: 599ms	remaining: 391ms
328:	learn: 0.0128005	total: 601ms	remaining: 389ms
329:	learn: 0.0128005	total: 632ms	remaining: 406ms
330:	learn: 0.0128004	total: 633ms	remaining: 404ms
331:	learn: 0.0128004	total: 635ms	remaining: 401ms
332:	learn: 0.0128004	total: 636ms	remaining: 399ms
333:	learn: 0.0128003	total: 637ms	remaining: 397ms
334:	learn: 0.0128001	total: 639ms	remaining: 395ms
335:	learn: 0.0128001	total: 640ms	remaining: 392ms
336:	learn: 0.0128000	total: 641ms	remaining: 390ms
337:	learn: 0.0127999	total: 643ms	remaining: 388ms
338:	learn: 0.0128000	total: 644ms	remaining: 386ms
339:	learn: 0.0127996	total: 646ms	remaining: 384ms
340:	learn: 0.0127996	total: 647ms	remaining: 381ms
341:	learn: 0.0127996	total: 648ms	remaining: 379ms
342:	learn: 0.0127995	total: 650ms	remaining: 377ms
343:	learn: 0.0127996	total: 651ms	remaining: 375ms
344:	learn: 0.0127995	total: 652ms	remaining: 373ms
345:	learn: 0.0127995	total: 654ms	remaining: 370ms
346:	learn: 0.0127995	total: 655ms	remaining: 368ms
347:	learn: 0.0127993	total: 657ms	remaining: 366ms
348:	learn: 0.0127991	total: 658ms	remaining: 364ms
349:	learn: 0.0127990	total: 659ms	remaining: 362ms
350:	learn: 0.0127990	total: 661ms	remaining: 360ms
351:	learn: 0.0127989	total: 662ms	remaining: 357ms
352:	learn: 0.0127990	total: 663ms	remaining: 355ms
353:	learn: 0.0127989	total: 665ms	remaining: 353ms
354:	learn: 0.0127989	total: 666ms	remaining: 351ms
355:	learn: 0.0127989	total: 667ms	remaining: 349ms
356:	learn: 0.0127988	total: 668ms	remaining: 346ms
357:	learn: 0.0127988	total: 670ms	remaining: 344ms
358:	learn: 0.0127987	total: 671ms	remaining: 342ms
359:	learn: 0.0127987	total: 673ms	remaining: 340ms
360:	learn: 0.0127985	total: 674ms	remaining: 338ms
361:	learn: 0.0127985	total: 676ms	remaining: 336ms
362:	learn: 0.0127985	total: 677ms	remaining: 334ms
363:	learn: 0.0127984	total: 679ms	remaining: 332ms
364:	learn: 0.0127983	total: 680ms	remaining: 330ms
365:	learn: 0.0127983	total: 681ms	remaining: 328ms
366:	learn: 0.0127982	total: 683ms	remaining: 326ms
367:	learn: 0.0127981	total: 684ms	remaining: 323ms
368:	learn: 0.0127979	total: 686ms	remaining: 321ms
369:	learn: 0.0127978	total: 687ms	remaining: 319ms
370:	learn: 0.0127977	total: 688ms	remaining: 317ms
371:	learn: 0.0127973	total: 690ms	remaining: 315ms
372:	learn: 0.0127973	total: 691ms	remaining: 313ms
373:	learn: 0.0127974	total: 693ms	remaining: 311ms
374:	learn: 0.0127973	total: 694ms	remaining: 309ms
375:	learn: 0.0127974	total: 696ms	remaining: 307ms
376:	learn: 0.0127973	total: 697ms	remaining: 305ms
377:	learn: 0.0127972	total: 699ms	remaining: 303ms
378:	learn: 0.0127968	total: 700ms	remaining: 301ms
379:	learn: 0.0127968	total: 702ms	remaining: 299ms
380:	learn: 0.0127967	total: 703ms	remaining: 297ms
381:	learn: 0.0127967	total: 705ms	remaining: 295ms
382:	learn: 0.0127967	total: 706ms	remaining: 293ms
383:	learn: 0.0127966	total: 708ms	remaining: 291ms
384:	learn: 0.0127966	total: 710ms	remaining: 289ms
385:	learn: 0.0127966	total: 711ms	remaining: 287ms
386:	learn: 0.0127966	total: 713ms	remaining: 286ms
387:	learn: 0.0127965	total: 715ms	remaining: 284ms
388:	learn: 0.0127962	total: 717ms	remaining: 282ms
389:	learn: 0.0127963	total: 718ms	remaining: 280ms
390:	learn: 0.0127962	total: 720ms	remaining: 278ms
391:	learn: 0.0127961	total: 721ms	remaining: 276ms
392:	learn: 0.0127958	total: 723ms	remaining: 274ms
393:	learn: 0.0127958	total: 724ms	remaining: 272ms
394:	learn: 0.0127958	total: 726ms	remaining: 270ms
395:	learn: 0.0127957	total: 728ms	remaining: 268ms
396:	learn: 0.0127953	total: 729ms	remaining: 266ms
397:	learn: 0.0127951	total: 731ms	remaining: 264ms
398:	learn: 0.0127951	total: 732ms	remaining: 262ms
399:	learn: 0.0127951	total: 734ms	remaining: 260ms
400:	learn: 0.0127949	total: 735ms	remaining: 259ms
401:	learn: 0.0127947	total: 737ms	remaining: 257ms
402:	learn: 0.0127728	total: 739ms	remaining: 255ms
403:	learn: 0.0127728	total: 741ms	remaining: 253ms
404:	learn: 0.0127527	total: 743ms	remaining: 251ms
405:	learn: 0.0127341	total: 748ms	remaining: 251ms
406:	learn: 0.0127341	total: 750ms	remaining: 249ms
407:	learn: 0.0127338	total: 751ms	remaining: 247ms
408:	learn: 0.0127337	total: 753ms	remaining: 245ms
409:	learn: 0.0127336	total: 754ms	remaining: 243ms
410:	learn: 0.0127331	total: 756ms	remaining: 241ms
411:	learn: 0.0127332	total: 757ms	remaining: 239ms
412:	learn: 0.0127328	total: 759ms	remaining: 237ms
413:	learn: 0.0127324	total: 760ms	remaining: 235ms
414:	learn: 0.0127324	total: 762ms	remaining: 233ms
415:	learn: 0.0127323	total: 763ms	remaining: 231ms
416:	learn: 0.0127322	total: 765ms	remaining: 229ms
417:	learn: 0.0127321	total: 767ms	remaining: 227ms
418:	learn: 0.0127321	total: 778ms	remaining: 228ms
419:	learn: 0.0127320	total: 779ms	remaining: 226ms
420:	learn: 0.0127319	total: 781ms	remaining: 224ms
421:	learn: 0.0127317	total: 782ms	remaining: 222ms
422:	learn: 0.0127316	total: 783ms	remaining: 220ms
423:	learn: 0.0127315	total: 785ms	remaining: 218ms
424:	learn: 0.0127288	total: 787ms	remaining: 217ms
425:	learn: 0.0127011	total: 788ms	remaining: 215ms
426:	learn: 0.0126929	total: 790ms	remaining: 213ms
427:	learn: 0.0126928	total: 792ms	remaining: 211ms
428:	learn: 0.0126897	total: 794ms	remaining: 209ms
429:	learn: 0.0126890	total: 795ms	remaining: 207ms
430:	learn: 0.0126641	total: 797ms	remaining: 205ms
431:	learn: 0.0126621	total: 799ms	remaining: 204ms
432:	learn: 0.0126618	total: 801ms	remaining: 202ms
433:	learn: 0.0126398	total: 803ms	remaining: 200ms
434:	learn: 0.0126214	total: 805ms	remaining: 198ms
435:	learn: 0.0126213	total: 807ms	remaining: 196ms
436:	learn: 0.0126038	total: 808ms	remaining: 194ms
437:	learn: 0.0126011	total: 810ms	remaining: 192ms
438:	learn: 0.0125017	total: 812ms	remaining: 191ms
439:	learn: 0.0124001	total: 814ms	remaining: 189ms
440:	learn: 0.0123194	total: 815ms	remaining: 187ms
441:	learn: 0.0122422	total: 817ms	remaining: 185ms
442:	learn: 0.0121909	total: 818ms	remaining: 183ms
443:	learn: 0.0121024	total: 820ms	remaining: 181ms
444:	learn: 0.0120892	total: 822ms	remaining: 179ms
445:	learn: 0.0119968	total: 824ms	remaining: 177ms
446:	learn: 0.0119909	total: 825ms	remaining: 175ms
447:	learn: 0.0119837	total: 827ms	remaining: 174ms
448:	learn: 0.0119767	total: 829ms	remaining: 172ms
449:	learn: 0.0119759	total: 831ms	remaining: 170ms
450:	learn: 0.0119074	total: 832ms	remaining: 168ms
451:	learn: 0.0119006	total: 834ms	remaining: 166ms
452:	learn: 0.0118948	total: 836ms	remaining: 164ms
453:	learn: 0.0118939	total: 838ms	remaining: 162ms
454:	learn: 0.0118792	total: 839ms	remaining: 160ms
455:	learn: 0.0118791	total: 841ms	remaining: 159ms
456:	learn: 0.0118659	total: 843ms	remaining: 157ms
457:	learn: 0.0118649	total: 844ms	remaining: 155ms
458:	learn: 0.0118041	total: 847ms	remaining: 153ms
459:	learn: 0.0117229	total: 848ms	remaining: 151ms
460:	learn: 0.0116718	total: 850ms	remaining: 149ms
461:	learn: 0.0116056	total: 852ms	remaining: 148ms
462:	learn: 0.0115993	total: 854ms	remaining: 146ms
463:	learn: 0.0115887	total: 856ms	remaining: 144ms
464:	learn: 0.0115881	total: 857ms	remaining: 142ms
465:	learn: 0.0115246	total: 860ms	remaining: 140ms
466:	learn: 0.0114815	total: 862ms	remaining: 138ms
467:	learn: 0.0114261	total: 863ms	remaining: 137ms
468:	learn: 0.0114099	total: 865ms	remaining: 135ms
469:	learn: 0.0113692	total: 867ms	remaining: 133ms
470:	learn: 0.0113452	total: 869ms	remaining: 131ms
471:	learn: 0.0113345	total: 870ms	remaining: 129ms
472:	learn: 0.0112992	total: 872ms	remaining: 127ms
473:	learn: 0.0112641	total: 874ms	remaining: 125ms
474:	learn: 0.0111935	total: 875ms	remaining: 123ms
475:	learn: 0.0110853	total: 877ms	remaining: 122ms
476:	learn: 0.0110244	total: 878ms	remaining: 120ms
477:	learn: 0.0110140	total: 880ms	remaining: 118ms
478:	learn: 0.0110035	total: 882ms	remaining: 116ms
479:	learn: 0.0109747	total: 883ms	remaining: 114ms
480:	learn: 0.0109109	total: 885ms	remaining: 112ms
481:	learn: 0.0108526	total: 887ms	remaining: 110ms
482:	learn: 0.0108403	total: 889ms	remaining: 109ms
483:	learn: 0.0107648	total: 891ms	remaining: 107ms
484:	learn: 0.0107125	total: 893ms	remaining: 105ms
485:	learn: 0.0106581	total: 894ms	remaining: 103ms
486:	learn: 0.0106471	total: 896ms	remaining: 101ms
487:	learn: 0.0105788	total: 898ms	remaining: 99.3ms
488:	learn: 0.0105530	total: 900ms	remaining: 97.5ms
489:	learn: 0.0104939	total: 901ms	remaining: 95.6ms
490:	learn: 0.0104355	total: 903ms	remaining: 93.8ms
491:	learn: 0.0103770	total: 905ms	remaining: 92ms
492:	learn: 0.0103578	total: 907ms	remaining: 90.1ms
493:	learn: 0.0103487	total: 909ms	remaining: 88.3ms
494:	learn: 0.0103007	total: 911ms	remaining: 86.5ms
495:	learn: 0.0102694	total: 912ms	remaining: 84.6ms
496:	learn: 0.0102302	total: 914ms	remaining: 82.8ms
497:	learn: 0.0101597	total: 916ms	remaining: 80.9ms
498:	learn: 0.0101595	total: 917ms	remaining: 79ms
499:	learn: 0.0101159	total: 919ms	remaining: 77.2ms
500:	learn: 0.0100841	total: 921ms	remaining: 75.4ms
501:	learn: 0.0100769	total: 923ms	remaining: 73.5ms
502:	learn: 0.0100403	total: 924ms	remaining: 71.7ms
503:	learn: 0.0099859	total: 926ms	remaining: 69.8ms
504:	learn: 0.0099785	total: 928ms	remaining: 68ms
505:	learn: 0.0099604	total: 929ms	remaining: 66.1ms
506:	learn: 0.0099283	total: 931ms	remaining: 64.3ms
507:	learn: 0.0098655	total: 932ms	remaining: 62.4ms
508:	learn: 0.0098556	total: 934ms	remaining: 60.5ms
509:	learn: 0.0098494	total: 936ms	remaining: 58.7ms
510:	learn: 0.0098446	total: 937ms	remaining: 56.9ms
511:	learn: 0.0098446	total: 939ms	remaining: 55ms
512:	learn: 0.0098169	total: 950ms	remaining: 53.7ms
513:	learn: 0.0098078	total: 953ms	remaining: 51.9ms
514:	learn: 0.0097987	total: 955ms	remaining: 50ms
515:	learn: 0.0097435	total: 957ms	remaining: 48.2ms
516:	learn: 0.0097349	total: 959ms	remaining: 46.4ms
517:	learn: 0.0096584	total: 960ms	remaining: 44.5ms
518:	learn: 0.0096097	total: 962ms	remaining: 42.6ms
519:	learn: 0.0095626	total: 964ms	remaining: 40.8ms
520:	learn: 0.0095143	total: 966ms	remaining: 38.9ms
521:	learn: 0.0094755	total: 967ms	remaining: 37.1ms
522:	learn: 0.0094430	total: 969ms	remaining: 35.2ms
523:	learn: 0.0093825	total: 970ms	remaining: 33.3ms
524:	learn: 0.0093247	total: 972ms	remaining: 31.5ms
525:	learn: 0.0092507	total: 973ms	remaining: 29.6ms
526:	learn: 0.0091831	total: 975ms	remaining: 27.8ms
527:	learn: 0.0091286	total: 977ms	remaining: 25.9ms
528:	learn: 0.0090992	total: 980ms	remaining: 24.1ms
529:	learn: 0.0090759	total: 982ms	remaining: 22.2ms
530:	learn: 0.0090320	total: 984ms	remaining: 20.4ms
531:	learn: 0.0089988	total: 986ms	remaining: 18.5ms
532:	learn: 0.0089524	total: 988ms	remaining: 16.7ms
533:	learn: 0.0089074	total: 990ms	remaining: 14.8ms
534:	learn: 0.0088658	total: 992ms	remaining: 13ms
535:	learn: 0.0088185	total: 994ms	remaining: 11.1ms
536:	learn: 0.0087800	total: 995ms	remaining: 9.27ms
537:	learn: 0.0087535	total: 997ms	remaining: 7.41ms
538:	learn: 0.0087226	total: 999ms	remaining: 5.56ms
539:	learn: 0.0086846	total: 1s	remaining: 3.71ms
540:	learn: 0.0086564	total: 1s	remaining: 1.85ms
541:	learn: 0.0086395	total: 1s	remaining: 0us
0:	learn: 0.5468908	total: 1.77ms	remaining: 959ms
1:	learn: 0.4578622	total: 3.42ms	remaining: 925ms
2:	learn: 0.3865623	total: 5.22ms	remaining: 938ms
3:	learn: 0.3464703	total: 7.01ms	remaining: 943ms
4:	learn: 0.3162576	total: 8.89ms	remaining: 955ms
5:	learn: 0.2926005	total: 10.6ms	remaining: 949ms
6:	learn: 0.2782276	total: 12.2ms	remaining: 936ms
7:	learn: 0.2633169	total: 13.8ms	remaining: 924ms
8:	learn: 0.2553097	total: 15.4ms	remaining: 914ms
9:	learn: 0.2428444	total: 17ms	remaining: 905ms
10:	learn: 0.2347012	total: 18.5ms	remaining: 893ms
11:	learn: 0.2230099	total: 20.2ms	remaining: 891ms
12:	learn: 0.2171597	total: 21.7ms	remaining: 885ms
13:	learn: 0.2131791	total: 23.3ms	remaining: 879ms
14:	learn: 0.2083774	total: 24.9ms	remaining: 876ms
15:	learn: 0.2033750	total: 26.5ms	remaining: 870ms
16:	learn: 0.1988835	total: 27.9ms	remaining: 863ms
17:	learn: 0.1919722	total: 29.4ms	remaining: 856ms
18:	learn: 0.1861987	total: 30.9ms	remaining: 850ms
19:	learn: 0.1832602	total: 32.4ms	remaining: 847ms
20:	learn: 0.1789536	total: 33.9ms	remaining: 841ms
21:	learn: 0.1748499	total: 35.6ms	remaining: 841ms
22:	learn: 0.1710832	total: 37.7ms	remaining: 852ms
23:	learn: 0.1667124	total: 39.6ms	remaining: 855ms
24:	learn: 0.1635775	total: 41.3ms	remaining: 854ms
25:	learn: 0.1601955	total: 42.9ms	remaining: 851ms
26:	learn: 0.1578862	total: 44.8ms	remaining: 854ms
27:	learn: 0.1536640	total: 46.6ms	remaining: 855ms
28:	learn: 0.1510426	total: 48.4ms	remaining: 856ms
29:	learn: 0.1486285	total: 50.2ms	remaining: 857ms
30:	learn: 0.1442159	total: 51.8ms	remaining: 854ms
31:	learn: 0.1402550	total: 53.4ms	remaining: 852ms
32:	learn: 0.1394748	total: 55.2ms	remaining: 851ms
33:	learn: 0.1363015	total: 56.8ms	remaining: 848ms
34:	learn: 0.1339865	total: 58.3ms	remaining: 845ms
35:	learn: 0.1315227	total: 59.9ms	remaining: 842ms
36:	learn: 0.1310664	total: 61.3ms	remaining: 837ms
37:	learn: 0.1309319	total: 63.1ms	remaining: 837ms
38:	learn: 0.1299522	total: 64.7ms	remaining: 835ms
39:	learn: 0.1272185	total: 66.5ms	remaining: 835ms
40:	learn: 0.1240670	total: 68.4ms	remaining: 836ms
41:	learn: 0.1237492	total: 70.1ms	remaining: 835ms
42:	learn: 0.1200190	total: 71.7ms	remaining: 832ms
43:	learn: 0.1173104	total: 73.2ms	remaining: 829ms
44:	learn: 0.1168364	total: 75.2ms	remaining: 830ms
45:	learn: 0.1156497	total: 76.8ms	remaining: 828ms
46:	learn: 0.1148716	total: 78.5ms	remaining: 827ms
47:	learn: 0.1147085	total: 80.1ms	remaining: 824ms
48:	learn: 0.1107648	total: 81.7ms	remaining: 822ms
49:	learn: 0.1100461	total: 83.3ms	remaining: 820ms
50:	learn: 0.1088425	total: 84.9ms	remaining: 817ms
51:	learn: 0.1086502	total: 86.6ms	remaining: 816ms
52:	learn: 0.1059847	total: 88.1ms	remaining: 813ms
53:	learn: 0.1031516	total: 89.7ms	remaining: 810ms
54:	learn: 0.1025943	total: 91.1ms	remaining: 807ms
55:	learn: 0.1019146	total: 92.7ms	remaining: 805ms
56:	learn: 0.0996610	total: 94.4ms	remaining: 803ms
57:	learn: 0.0969921	total: 96.1ms	remaining: 802ms
58:	learn: 0.0940507	total: 98.1ms	remaining: 803ms
59:	learn: 0.0939873	total: 99.8ms	remaining: 802ms
60:	learn: 0.0901594	total: 102ms	remaining: 800ms
61:	learn: 0.0892996	total: 103ms	remaining: 797ms
62:	learn: 0.0885337	total: 104ms	remaining: 794ms
63:	learn: 0.0877932	total: 106ms	remaining: 792ms
64:	learn: 0.0853710	total: 108ms	remaining: 790ms
65:	learn: 0.0852997	total: 109ms	remaining: 788ms
66:	learn: 0.0843296	total: 111ms	remaining: 785ms
67:	learn: 0.0829407	total: 112ms	remaining: 783ms
68:	learn: 0.0824438	total: 114ms	remaining: 781ms
69:	learn: 0.0799699	total: 115ms	remaining: 778ms
70:	learn: 0.0795065	total: 117ms	remaining: 776ms
71:	learn: 0.0783406	total: 118ms	remaining: 774ms
72:	learn: 0.0768417	total: 120ms	remaining: 771ms
73:	learn: 0.0749401	total: 122ms	remaining: 768ms
74:	learn: 0.0746843	total: 123ms	remaining: 767ms
75:	learn: 0.0746329	total: 125ms	remaining: 766ms
76:	learn: 0.0745841	total: 127ms	remaining: 765ms
77:	learn: 0.0745382	total: 128ms	remaining: 764ms
78:	learn: 0.0744945	total: 130ms	remaining: 764ms
79:	learn: 0.0744532	total: 132ms	remaining: 762ms
80:	learn: 0.0744137	total: 134ms	remaining: 762ms
81:	learn: 0.0743761	total: 135ms	remaining: 760ms
82:	learn: 0.0743403	total: 137ms	remaining: 759ms
83:	learn: 0.0743064	total: 139ms	remaining: 758ms
84:	learn: 0.0733659	total: 141ms	remaining: 756ms
85:	learn: 0.0730162	total: 142ms	remaining: 755ms
86:	learn: 0.0723748	total: 144ms	remaining: 755ms
87:	learn: 0.0723406	total: 146ms	remaining: 753ms
88:	learn: 0.0711362	total: 147ms	remaining: 750ms
89:	learn: 0.0711046	total: 149ms	remaining: 748ms
90:	learn: 0.0710754	total: 150ms	remaining: 746ms
91:	learn: 0.0710477	total: 152ms	remaining: 743ms
92:	learn: 0.0710212	total: 154ms	remaining: 742ms
93:	learn: 0.0709959	total: 155ms	remaining: 740ms
94:	learn: 0.0709699	total: 157ms	remaining: 739ms
95:	learn: 0.0709465	total: 159ms	remaining: 738ms
96:	learn: 0.0702222	total: 161ms	remaining: 736ms
97:	learn: 0.0687564	total: 162ms	remaining: 735ms
98:	learn: 0.0683721	total: 164ms	remaining: 733ms
99:	learn: 0.0667459	total: 166ms	remaining: 732ms
100:	learn: 0.0664506	total: 167ms	remaining: 730ms
101:	learn: 0.0656518	total: 169ms	remaining: 729ms
102:	learn: 0.0640049	total: 171ms	remaining: 728ms
103:	learn: 0.0638163	total: 173ms	remaining: 728ms
104:	learn: 0.0629013	total: 174ms	remaining: 726ms
105:	learn: 0.0618966	total: 176ms	remaining: 724ms
106:	learn: 0.0607090	total: 178ms	remaining: 722ms
107:	learn: 0.0599650	total: 179ms	remaining: 720ms
108:	learn: 0.0589865	total: 181ms	remaining: 718ms
109:	learn: 0.0577390	total: 182ms	remaining: 716ms
110:	learn: 0.0571262	total: 184ms	remaining: 714ms
111:	learn: 0.0569731	total: 186ms	remaining: 712ms
112:	learn: 0.0560623	total: 187ms	remaining: 711ms
113:	learn: 0.0550864	total: 189ms	remaining: 710ms
114:	learn: 0.0542160	total: 191ms	remaining: 708ms
115:	learn: 0.0536780	total: 193ms	remaining: 707ms
116:	learn: 0.0531283	total: 194ms	remaining: 705ms
117:	learn: 0.0528588	total: 196ms	remaining: 703ms
118:	learn: 0.0523803	total: 197ms	remaining: 702ms
119:	learn: 0.0516176	total: 199ms	remaining: 700ms
120:	learn: 0.0512111	total: 200ms	remaining: 697ms
121:	learn: 0.0503060	total: 202ms	remaining: 695ms
122:	learn: 0.0498376	total: 203ms	remaining: 692ms
123:	learn: 0.0492665	total: 205ms	remaining: 691ms
124:	learn: 0.0490950	total: 207ms	remaining: 689ms
125:	learn: 0.0487577	total: 208ms	remaining: 687ms
126:	learn: 0.0480404	total: 210ms	remaining: 685ms
127:	learn: 0.0475738	total: 211ms	remaining: 683ms
128:	learn: 0.0472018	total: 213ms	remaining: 681ms
129:	learn: 0.0468656	total: 215ms	remaining: 680ms
130:	learn: 0.0466925	total: 216ms	remaining: 679ms
131:	learn: 0.0465354	total: 221ms	remaining: 687ms
132:	learn: 0.0457748	total: 223ms	remaining: 686ms
133:	learn: 0.0450133	total: 225ms	remaining: 685ms
134:	learn: 0.0447048	total: 227ms	remaining: 684ms
135:	learn: 0.0435328	total: 229ms	remaining: 682ms
136:	learn: 0.0433060	total: 230ms	remaining: 681ms
137:	learn: 0.0427765	total: 232ms	remaining: 679ms
138:	learn: 0.0417784	total: 234ms	remaining: 678ms
139:	learn: 0.0416249	total: 236ms	remaining: 676ms
140:	learn: 0.0413205	total: 237ms	remaining: 674ms
141:	learn: 0.0411232	total: 239ms	remaining: 672ms
142:	learn: 0.0407373	total: 240ms	remaining: 670ms
143:	learn: 0.0403768	total: 242ms	remaining: 668ms
144:	learn: 0.0402613	total: 243ms	remaining: 666ms
145:	learn: 0.0400430	total: 245ms	remaining: 664ms
146:	learn: 0.0397722	total: 246ms	remaining: 662ms
147:	learn: 0.0390328	total: 249ms	remaining: 662ms
148:	learn: 0.0389967	total: 250ms	remaining: 661ms
149:	learn: 0.0389658	total: 252ms	remaining: 659ms
150:	learn: 0.0381285	total: 253ms	remaining: 656ms
151:	learn: 0.0379796	total: 255ms	remaining: 654ms
152:	learn: 0.0376523	total: 257ms	remaining: 653ms
153:	learn: 0.0372313	total: 258ms	remaining: 651ms
154:	learn: 0.0366888	total: 260ms	remaining: 649ms
155:	learn: 0.0366564	total: 261ms	remaining: 646ms
156:	learn: 0.0366329	total: 263ms	remaining: 644ms
157:	learn: 0.0366063	total: 264ms	remaining: 642ms
158:	learn: 0.0365031	total: 266ms	remaining: 641ms
159:	learn: 0.0363580	total: 268ms	remaining: 639ms
160:	learn: 0.0359364	total: 269ms	remaining: 637ms
161:	learn: 0.0351734	total: 271ms	remaining: 635ms
162:	learn: 0.0349152	total: 272ms	remaining: 633ms
163:	learn: 0.0346569	total: 274ms	remaining: 631ms
164:	learn: 0.0345198	total: 275ms	remaining: 629ms
165:	learn: 0.0344049	total: 277ms	remaining: 627ms
166:	learn: 0.0338769	total: 279ms	remaining: 626ms
167:	learn: 0.0335232	total: 281ms	remaining: 625ms
168:	learn: 0.0333747	total: 282ms	remaining: 624ms
169:	learn: 0.0333012	total: 284ms	remaining: 622ms
170:	learn: 0.0331501	total: 286ms	remaining: 620ms
171:	learn: 0.0325763	total: 287ms	remaining: 618ms
172:	learn: 0.0325028	total: 289ms	remaining: 616ms
173:	learn: 0.0324793	total: 291ms	remaining: 615ms
174:	learn: 0.0320614	total: 292ms	remaining: 613ms
175:	learn: 0.0315420	total: 294ms	remaining: 611ms
176:	learn: 0.0312260	total: 296ms	remaining: 609ms
177:	learn: 0.0308888	total: 297ms	remaining: 608ms
178:	learn: 0.0304593	total: 299ms	remaining: 605ms
179:	learn: 0.0302008	total: 300ms	remaining: 603ms
180:	learn: 0.0301753	total: 302ms	remaining: 602ms
181:	learn: 0.0297250	total: 303ms	remaining: 600ms
182:	learn: 0.0297070	total: 305ms	remaining: 599ms
183:	learn: 0.0296879	total: 307ms	remaining: 598ms
184:	learn: 0.0295479	total: 309ms	remaining: 596ms
185:	learn: 0.0292720	total: 311ms	remaining: 595ms
186:	learn: 0.0289721	total: 313ms	remaining: 594ms
187:	learn: 0.0289540	total: 314ms	remaining: 592ms
188:	learn: 0.0285472	total: 316ms	remaining: 590ms
189:	learn: 0.0284460	total: 318ms	remaining: 589ms
190:	learn: 0.0282613	total: 319ms	remaining: 587ms
191:	learn: 0.0282449	total: 321ms	remaining: 585ms
192:	learn: 0.0281882	total: 323ms	remaining: 584ms
193:	learn: 0.0280926	total: 325ms	remaining: 582ms
194:	learn: 0.0279306	total: 326ms	remaining: 581ms
195:	learn: 0.0274539	total: 328ms	remaining: 579ms
196:	learn: 0.0271848	total: 330ms	remaining: 577ms
197:	learn: 0.0269323	total: 331ms	remaining: 575ms
198:	learn: 0.0265646	total: 333ms	remaining: 574ms
199:	learn: 0.0264734	total: 334ms	remaining: 572ms
200:	learn: 0.0264309	total: 336ms	remaining: 570ms
201:	learn: 0.0264161	total: 338ms	remaining: 568ms
202:	learn: 0.0263698	total: 339ms	remaining: 566ms
203:	learn: 0.0263573	total: 341ms	remaining: 564ms
204:	learn: 0.0261210	total: 342ms	remaining: 562ms
205:	learn: 0.0260418	total: 344ms	remaining: 560ms
206:	learn: 0.0259628	total: 345ms	remaining: 559ms
207:	learn: 0.0256136	total: 347ms	remaining: 557ms
208:	learn: 0.0254353	total: 348ms	remaining: 555ms
209:	learn: 0.0252001	total: 350ms	remaining: 553ms
210:	learn: 0.0251651	total: 351ms	remaining: 551ms
211:	learn: 0.0249981	total: 353ms	remaining: 550ms
212:	learn: 0.0249624	total: 355ms	remaining: 548ms
213:	learn: 0.0248256	total: 356ms	remaining: 546ms
214:	learn: 0.0247270	total: 358ms	remaining: 544ms
215:	learn: 0.0245691	total: 359ms	remaining: 542ms
216:	learn: 0.0245571	total: 361ms	remaining: 540ms
217:	learn: 0.0243308	total: 362ms	remaining: 538ms
218:	learn: 0.0241462	total: 364ms	remaining: 537ms
219:	learn: 0.0240342	total: 365ms	remaining: 535ms
220:	learn: 0.0239780	total: 367ms	remaining: 533ms
221:	learn: 0.0239343	total: 369ms	remaining: 531ms
222:	learn: 0.0237052	total: 370ms	remaining: 529ms
223:	learn: 0.0236621	total: 372ms	remaining: 528ms
224:	learn: 0.0234992	total: 373ms	remaining: 526ms
225:	learn: 0.0234590	total: 375ms	remaining: 524ms
226:	learn: 0.0231772	total: 376ms	remaining: 522ms
227:	learn: 0.0231042	total: 378ms	remaining: 520ms
228:	learn: 0.0230617	total: 379ms	remaining: 518ms
229:	learn: 0.0230322	total: 381ms	remaining: 517ms
230:	learn: 0.0229812	total: 383ms	remaining: 515ms
231:	learn: 0.0227854	total: 384ms	remaining: 514ms
232:	learn: 0.0225791	total: 386ms	remaining: 512ms
233:	learn: 0.0225686	total: 388ms	remaining: 510ms
234:	learn: 0.0225238	total: 389ms	remaining: 509ms
235:	learn: 0.0224340	total: 391ms	remaining: 507ms
236:	learn: 0.0223763	total: 393ms	remaining: 506ms
237:	learn: 0.0220070	total: 395ms	remaining: 504ms
238:	learn: 0.0218782	total: 397ms	remaining: 503ms
239:	learn: 0.0216440	total: 399ms	remaining: 502ms
240:	learn: 0.0216069	total: 401ms	remaining: 500ms
241:	learn: 0.0213543	total: 402ms	remaining: 499ms
242:	learn: 0.0213280	total: 404ms	remaining: 497ms
243:	learn: 0.0212160	total: 405ms	remaining: 495ms
244:	learn: 0.0212059	total: 407ms	remaining: 494ms
245:	learn: 0.0210016	total: 409ms	remaining: 492ms
246:	learn: 0.0209794	total: 410ms	remaining: 490ms
247:	learn: 0.0208154	total: 412ms	remaining: 488ms
248:	learn: 0.0206123	total: 413ms	remaining: 487ms
249:	learn: 0.0204407	total: 415ms	remaining: 485ms
250:	learn: 0.0202847	total: 417ms	remaining: 483ms
251:	learn: 0.0201157	total: 419ms	remaining: 482ms
252:	learn: 0.0200505	total: 420ms	remaining: 480ms
253:	learn: 0.0199922	total: 422ms	remaining: 478ms
254:	learn: 0.0199833	total: 423ms	remaining: 477ms
255:	learn: 0.0198184	total: 425ms	remaining: 475ms
256:	learn: 0.0196724	total: 427ms	remaining: 473ms
257:	learn: 0.0195811	total: 428ms	remaining: 471ms
258:	learn: 0.0194601	total: 430ms	remaining: 469ms
259:	learn: 0.0193426	total: 431ms	remaining: 468ms
260:	learn: 0.0192320	total: 433ms	remaining: 466ms
261:	learn: 0.0190952	total: 434ms	remaining: 464ms
262:	learn: 0.0188896	total: 436ms	remaining: 462ms
263:	learn: 0.0187309	total: 437ms	remaining: 460ms
264:	learn: 0.0186170	total: 439ms	remaining: 459ms
265:	learn: 0.0184901	total: 440ms	remaining: 457ms
266:	learn: 0.0182357	total: 443ms	remaining: 456ms
267:	learn: 0.0180836	total: 452ms	remaining: 462ms
268:	learn: 0.0180773	total: 453ms	remaining: 460ms
269:	learn: 0.0180760	total: 455ms	remaining: 458ms
270:	learn: 0.0180734	total: 456ms	remaining: 456ms
271:	learn: 0.0180716	total: 458ms	remaining: 454ms
272:	learn: 0.0180694	total: 459ms	remaining: 452ms
273:	learn: 0.0180672	total: 461ms	remaining: 450ms
274:	learn: 0.0179101	total: 462ms	remaining: 449ms
275:	learn: 0.0178022	total: 464ms	remaining: 447ms
276:	learn: 0.0178002	total: 465ms	remaining: 445ms
277:	learn: 0.0177983	total: 466ms	remaining: 443ms
278:	learn: 0.0177952	total: 468ms	remaining: 441ms
279:	learn: 0.0177921	total: 469ms	remaining: 439ms
280:	learn: 0.0177898	total: 471ms	remaining: 437ms
281:	learn: 0.0177628	total: 473ms	remaining: 436ms
282:	learn: 0.0176992	total: 475ms	remaining: 434ms
283:	learn: 0.0175834	total: 476ms	remaining: 433ms
284:	learn: 0.0175805	total: 478ms	remaining: 431ms
285:	learn: 0.0175696	total: 479ms	remaining: 429ms
286:	learn: 0.0175402	total: 481ms	remaining: 427ms
287:	learn: 0.0173484	total: 482ms	remaining: 425ms
288:	learn: 0.0173230	total: 484ms	remaining: 423ms
289:	learn: 0.0172952	total: 485ms	remaining: 422ms
290:	learn: 0.0172583	total: 487ms	remaining: 420ms
291:	learn: 0.0172244	total: 488ms	remaining: 418ms
292:	learn: 0.0172046	total: 490ms	remaining: 416ms
293:	learn: 0.0169912	total: 491ms	remaining: 414ms
294:	learn: 0.0168073	total: 493ms	remaining: 412ms
295:	learn: 0.0167113	total: 494ms	remaining: 411ms
296:	learn: 0.0165368	total: 496ms	remaining: 409ms
297:	learn: 0.0163828	total: 497ms	remaining: 407ms
298:	learn: 0.0163731	total: 499ms	remaining: 405ms
299:	learn: 0.0162207	total: 500ms	remaining: 404ms
300:	learn: 0.0162058	total: 502ms	remaining: 402ms
301:	learn: 0.0161679	total: 504ms	remaining: 400ms
302:	learn: 0.0161339	total: 506ms	remaining: 399ms
303:	learn: 0.0161254	total: 507ms	remaining: 397ms
304:	learn: 0.0160283	total: 509ms	remaining: 395ms
305:	learn: 0.0158764	total: 510ms	remaining: 394ms
306:	learn: 0.0158167	total: 512ms	remaining: 392ms
307:	learn: 0.0157625	total: 514ms	remaining: 390ms
308:	learn: 0.0157440	total: 516ms	remaining: 389ms
309:	learn: 0.0157271	total: 517ms	remaining: 387ms
310:	learn: 0.0156995	total: 519ms	remaining: 385ms
311:	learn: 0.0156606	total: 520ms	remaining: 384ms
312:	learn: 0.0156351	total: 522ms	remaining: 382ms
313:	learn: 0.0156098	total: 524ms	remaining: 380ms
314:	learn: 0.0155859	total: 525ms	remaining: 379ms
315:	learn: 0.0154920	total: 527ms	remaining: 377ms
316:	learn: 0.0154700	total: 528ms	remaining: 375ms
317:	learn: 0.0154185	total: 530ms	remaining: 373ms
318:	learn: 0.0154049	total: 532ms	remaining: 372ms
319:	learn: 0.0153910	total: 534ms	remaining: 370ms
320:	learn: 0.0153044	total: 535ms	remaining: 369ms
321:	learn: 0.0152905	total: 537ms	remaining: 367ms
322:	learn: 0.0152710	total: 539ms	remaining: 365ms
323:	learn: 0.0151382	total: 540ms	remaining: 364ms
324:	learn: 0.0150811	total: 542ms	remaining: 362ms
325:	learn: 0.0150601	total: 543ms	remaining: 360ms
326:	learn: 0.0150403	total: 545ms	remaining: 358ms
327:	learn: 0.0150304	total: 547ms	remaining: 357ms
328:	learn: 0.0148893	total: 548ms	remaining: 355ms
329:	learn: 0.0148681	total: 550ms	remaining: 353ms
330:	learn: 0.0148630	total: 552ms	remaining: 352ms
331:	learn: 0.0148312	total: 553ms	remaining: 350ms
332:	learn: 0.0148174	total: 555ms	remaining: 348ms
333:	learn: 0.0148054	total: 556ms	remaining: 347ms
334:	learn: 0.0148053	total: 558ms	remaining: 345ms
335:	learn: 0.0147690	total: 559ms	remaining: 343ms
336:	learn: 0.0147570	total: 561ms	remaining: 341ms
337:	learn: 0.0147441	total: 563ms	remaining: 340ms
338:	learn: 0.0146566	total: 564ms	remaining: 338ms
339:	learn: 0.0146517	total: 566ms	remaining: 336ms
340:	learn: 0.0146274	total: 568ms	remaining: 335ms
341:	learn: 0.0146103	total: 582ms	remaining: 341ms
342:	learn: 0.0145334	total: 584ms	remaining: 339ms
343:	learn: 0.0144821	total: 586ms	remaining: 337ms
344:	learn: 0.0144628	total: 587ms	remaining: 335ms
345:	learn: 0.0143249	total: 589ms	remaining: 333ms
346:	learn: 0.0142048	total: 590ms	remaining: 332ms
347:	learn: 0.0141832	total: 592ms	remaining: 330ms
348:	learn: 0.0141672	total: 594ms	remaining: 328ms
349:	learn: 0.0140526	total: 596ms	remaining: 327ms
350:	learn: 0.0139316	total: 598ms	remaining: 325ms
351:	learn: 0.0138569	total: 599ms	remaining: 324ms
352:	learn: 0.0138131	total: 601ms	remaining: 322ms
353:	learn: 0.0137983	total: 603ms	remaining: 320ms
354:	learn: 0.0137301	total: 604ms	remaining: 318ms
355:	learn: 0.0136951	total: 606ms	remaining: 317ms
356:	learn: 0.0136787	total: 607ms	remaining: 315ms
357:	learn: 0.0136155	total: 609ms	remaining: 313ms
358:	learn: 0.0135977	total: 611ms	remaining: 311ms
359:	learn: 0.0134934	total: 612ms	remaining: 310ms
360:	learn: 0.0134807	total: 614ms	remaining: 308ms
361:	learn: 0.0134713	total: 616ms	remaining: 306ms
362:	learn: 0.0134602	total: 617ms	remaining: 304ms
363:	learn: 0.0133938	total: 619ms	remaining: 302ms
364:	learn: 0.0133937	total: 620ms	remaining: 301ms
365:	learn: 0.0133792	total: 622ms	remaining: 299ms
366:	learn: 0.0133663	total: 623ms	remaining: 297ms
367:	learn: 0.0133573	total: 625ms	remaining: 296ms
368:	learn: 0.0133422	total: 627ms	remaining: 294ms
369:	learn: 0.0133307	total: 628ms	remaining: 292ms
370:	learn: 0.0133223	total: 630ms	remaining: 290ms
371:	learn: 0.0132889	total: 631ms	remaining: 288ms
372:	learn: 0.0132810	total: 633ms	remaining: 287ms
373:	learn: 0.0132732	total: 634ms	remaining: 285ms
374:	learn: 0.0132461	total: 636ms	remaining: 283ms
375:	learn: 0.0131935	total: 637ms	remaining: 281ms
376:	learn: 0.0131186	total: 639ms	remaining: 280ms
377:	learn: 0.0131106	total: 640ms	remaining: 278ms
378:	learn: 0.0129995	total: 642ms	remaining: 276ms
379:	learn: 0.0128910	total: 643ms	remaining: 274ms
380:	learn: 0.0128078	total: 645ms	remaining: 273ms
381:	learn: 0.0127471	total: 647ms	remaining: 271ms
382:	learn: 0.0126330	total: 648ms	remaining: 269ms
383:	learn: 0.0126261	total: 649ms	remaining: 267ms
384:	learn: 0.0126192	total: 651ms	remaining: 265ms
385:	learn: 0.0125314	total: 653ms	remaining: 264ms
386:	learn: 0.0125247	total: 655ms	remaining: 262ms
387:	learn: 0.0124292	total: 657ms	remaining: 261ms
388:	learn: 0.0122919	total: 659ms	remaining: 259ms
389:	learn: 0.0121787	total: 660ms	remaining: 257ms
390:	learn: 0.0121662	total: 662ms	remaining: 256ms
391:	learn: 0.0121014	total: 663ms	remaining: 254ms
392:	learn: 0.0120616	total: 665ms	remaining: 252ms
393:	learn: 0.0120265	total: 667ms	remaining: 250ms
394:	learn: 0.0119627	total: 668ms	remaining: 249ms
395:	learn: 0.0118898	total: 670ms	remaining: 247ms
396:	learn: 0.0117998	total: 672ms	remaining: 245ms
397:	learn: 0.0117753	total: 673ms	remaining: 244ms
398:	learn: 0.0116866	total: 674ms	remaining: 242ms
399:	learn: 0.0116342	total: 676ms	remaining: 240ms
400:	learn: 0.0115784	total: 678ms	remaining: 238ms
401:	learn: 0.0114965	total: 679ms	remaining: 237ms
402:	learn: 0.0114328	total: 681ms	remaining: 235ms
403:	learn: 0.0114144	total: 682ms	remaining: 233ms
404:	learn: 0.0113776	total: 684ms	remaining: 232ms
405:	learn: 0.0113560	total: 686ms	remaining: 230ms
406:	learn: 0.0113078	total: 688ms	remaining: 228ms
407:	learn: 0.0112610	total: 690ms	remaining: 227ms
408:	learn: 0.0112035	total: 692ms	remaining: 225ms
409:	learn: 0.0111563	total: 693ms	remaining: 223ms
410:	learn: 0.0111324	total: 695ms	remaining: 222ms
411:	learn: 0.0110600	total: 697ms	remaining: 220ms
412:	learn: 0.0110490	total: 699ms	remaining: 218ms
413:	learn: 0.0110159	total: 700ms	remaining: 217ms
414:	learn: 0.0109308	total: 702ms	remaining: 215ms
415:	learn: 0.0108879	total: 704ms	remaining: 213ms
416:	learn: 0.0108858	total: 705ms	remaining: 211ms
417:	learn: 0.0108756	total: 706ms	remaining: 210ms
418:	learn: 0.0108646	total: 708ms	remaining: 208ms
419:	learn: 0.0107935	total: 709ms	remaining: 206ms
420:	learn: 0.0107844	total: 711ms	remaining: 204ms
421:	learn: 0.0107193	total: 712ms	remaining: 203ms
422:	learn: 0.0106698	total: 714ms	remaining: 201ms
423:	learn: 0.0106185	total: 716ms	remaining: 199ms
424:	learn: 0.0105835	total: 717ms	remaining: 197ms
425:	learn: 0.0105834	total: 719ms	remaining: 196ms
426:	learn: 0.0105449	total: 720ms	remaining: 194ms
427:	learn: 0.0104805	total: 721ms	remaining: 192ms
428:	learn: 0.0104479	total: 723ms	remaining: 190ms
429:	learn: 0.0103968	total: 724ms	remaining: 189ms
430:	learn: 0.0103669	total: 726ms	remaining: 187ms
431:	learn: 0.0103615	total: 728ms	remaining: 185ms
432:	learn: 0.0103403	total: 729ms	remaining: 184ms
433:	learn: 0.0103222	total: 731ms	remaining: 182ms
434:	learn: 0.0102745	total: 732ms	remaining: 180ms
435:	learn: 0.0102531	total: 734ms	remaining: 178ms
436:	learn: 0.0102467	total: 736ms	remaining: 177ms
437:	learn: 0.0102461	total: 737ms	remaining: 175ms
438:	learn: 0.0102461	total: 738ms	remaining: 173ms
439:	learn: 0.0102461	total: 740ms	remaining: 171ms
440:	learn: 0.0102461	total: 741ms	remaining: 170ms
441:	learn: 0.0102461	total: 742ms	remaining: 168ms
442:	learn: 0.0102434	total: 744ms	remaining: 166ms
443:	learn: 0.0101994	total: 745ms	remaining: 165ms
444:	learn: 0.0101945	total: 747ms	remaining: 163ms
445:	learn: 0.0101775	total: 748ms	remaining: 161ms
446:	learn: 0.0101773	total: 750ms	remaining: 159ms
447:	learn: 0.0101773	total: 751ms	remaining: 158ms
448:	learn: 0.0101762	total: 753ms	remaining: 156ms
449:	learn: 0.0101762	total: 754ms	remaining: 154ms
450:	learn: 0.0101738	total: 755ms	remaining: 152ms
451:	learn: 0.0101579	total: 757ms	remaining: 151ms
452:	learn: 0.0101576	total: 758ms	remaining: 149ms
453:	learn: 0.0101565	total: 760ms	remaining: 147ms
454:	learn: 0.0101539	total: 762ms	remaining: 146ms
455:	learn: 0.0101538	total: 763ms	remaining: 144ms
456:	learn: 0.0101434	total: 765ms	remaining: 142ms
457:	learn: 0.0101433	total: 766ms	remaining: 140ms
458:	learn: 0.0101433	total: 767ms	remaining: 139ms
459:	learn: 0.0101433	total: 769ms	remaining: 137ms
460:	learn: 0.0101432	total: 770ms	remaining: 135ms
461:	learn: 0.0101433	total: 772ms	remaining: 134ms
462:	learn: 0.0101430	total: 773ms	remaining: 132ms
463:	learn: 0.0101342	total: 775ms	remaining: 130ms
464:	learn: 0.0101304	total: 776ms	remaining: 129ms
465:	learn: 0.0101180	total: 778ms	remaining: 127ms
466:	learn: 0.0100807	total: 780ms	remaining: 125ms
467:	learn: 0.0100340	total: 781ms	remaining: 124ms
468:	learn: 0.0099760	total: 783ms	remaining: 122ms
469:	learn: 0.0099758	total: 785ms	remaining: 120ms
470:	learn: 0.0099681	total: 786ms	remaining: 119ms
471:	learn: 0.0099535	total: 788ms	remaining: 117ms
472:	learn: 0.0099447	total: 790ms	remaining: 115ms
473:	learn: 0.0099332	total: 791ms	remaining: 114ms
474:	learn: 0.0099333	total: 793ms	remaining: 112ms
475:	learn: 0.0099329	total: 794ms	remaining: 110ms
476:	learn: 0.0099328	total: 796ms	remaining: 108ms
477:	learn: 0.0099256	total: 797ms	remaining: 107ms
478:	learn: 0.0099251	total: 799ms	remaining: 105ms
479:	learn: 0.0099250	total: 802ms	remaining: 104ms
480:	learn: 0.0099249	total: 805ms	remaining: 102ms
481:	learn: 0.0099232	total: 807ms	remaining: 100ms
482:	learn: 0.0099232	total: 809ms	remaining: 98.8ms
483:	learn: 0.0099231	total: 810ms	remaining: 97.1ms
484:	learn: 0.0099231	total: 812ms	remaining: 95.5ms
485:	learn: 0.0099230	total: 814ms	remaining: 93.8ms
486:	learn: 0.0099224	total: 816ms	remaining: 92.2ms
487:	learn: 0.0099139	total: 819ms	remaining: 90.6ms
488:	learn: 0.0098999	total: 821ms	remaining: 89ms
489:	learn: 0.0098813	total: 824ms	remaining: 87.4ms
490:	learn: 0.0098642	total: 826ms	remaining: 85.8ms
491:	learn: 0.0098641	total: 827ms	remaining: 84.1ms
492:	learn: 0.0098640	total: 829ms	remaining: 82.4ms
493:	learn: 0.0098639	total: 831ms	remaining: 80.7ms
494:	learn: 0.0098535	total: 833ms	remaining: 79.1ms
495:	learn: 0.0098106	total: 835ms	remaining: 77.4ms
496:	learn: 0.0098106	total: 836ms	remaining: 75.7ms
497:	learn: 0.0098015	total: 838ms	remaining: 74.1ms
498:	learn: 0.0098014	total: 840ms	remaining: 72.4ms
499:	learn: 0.0097932	total: 841ms	remaining: 70.7ms
500:	learn: 0.0097577	total: 843ms	remaining: 69ms
501:	learn: 0.0097035	total: 845ms	remaining: 67.3ms
502:	learn: 0.0096456	total: 846ms	remaining: 65.6ms
503:	learn: 0.0096451	total: 848ms	remaining: 63.9ms
504:	learn: 0.0096299	total: 856ms	remaining: 62.7ms
505:	learn: 0.0096297	total: 857ms	remaining: 61ms
506:	learn: 0.0096246	total: 859ms	remaining: 59.3ms
507:	learn: 0.0096245	total: 860ms	remaining: 57.5ms
508:	learn: 0.0096244	total: 861ms	remaining: 55.8ms
509:	learn: 0.0096244	total: 862ms	remaining: 54.1ms
510:	learn: 0.0096244	total: 864ms	remaining: 52.4ms
511:	learn: 0.0096244	total: 865ms	remaining: 50.7ms
512:	learn: 0.0096244	total: 866ms	remaining: 49ms
513:	learn: 0.0096244	total: 868ms	remaining: 47.3ms
514:	learn: 0.0096244	total: 869ms	remaining: 45.5ms
515:	learn: 0.0096244	total: 870ms	remaining: 43.8ms
516:	learn: 0.0096244	total: 871ms	remaining: 42.1ms
517:	learn: 0.0096244	total: 873ms	remaining: 40.4ms
518:	learn: 0.0096244	total: 874ms	remaining: 38.7ms
519:	learn: 0.0096244	total: 875ms	remaining: 37ms
520:	learn: 0.0096244	total: 876ms	remaining: 35.3ms
521:	learn: 0.0096244	total: 878ms	remaining: 33.6ms
522:	learn: 0.0096244	total: 879ms	remaining: 31.9ms
523:	learn: 0.0096244	total: 880ms	remaining: 30.2ms
524:	learn: 0.0096244	total: 882ms	remaining: 28.6ms
525:	learn: 0.0096244	total: 883ms	remaining: 26.9ms
526:	learn: 0.0096244	total: 884ms	remaining: 25.2ms
527:	learn: 0.0096244	total: 886ms	remaining: 23.5ms
528:	learn: 0.0096244	total: 887ms	remaining: 21.8ms
529:	learn: 0.0096244	total: 888ms	remaining: 20.1ms
530:	learn: 0.0096244	total: 890ms	remaining: 18.4ms
531:	learn: 0.0096244	total: 891ms	remaining: 16.7ms
532:	learn: 0.0096244	total: 892ms	remaining: 15.1ms
533:	learn: 0.0096244	total: 894ms	remaining: 13.4ms
534:	learn: 0.0096244	total: 895ms	remaining: 11.7ms
535:	learn: 0.0096244	total: 896ms	remaining: 10ms
536:	learn: 0.0096244	total: 897ms	remaining: 8.36ms
537:	learn: 0.0096244	total: 899ms	remaining: 6.68ms
538:	learn: 0.0096244	total: 900ms	remaining: 5.01ms
539:	learn: 0.0096244	total: 901ms	remaining: 3.34ms
540:	learn: 0.0096244	total: 902ms	remaining: 1.67ms
541:	learn: 0.0096244	total: 904ms	remaining: 0us
Bagging Classifier Accuracy: 0.8967391304347826

X = newDf.drop('HeartDisease', axis=1)
y= newDf['HeartDisease']

X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=42)

#'iterations': 542, 'learning_rate': 0.21231296216531056, 'depth': 6, 'l2_leaf_reg': 13.675507885439055
cat_model = CatBoostClassifier(iterations=542, learning_rate=0.21231296216531056, depth=6, l2_leaf_reg=13.675507885439055)
cat_model.fit(X_train, y_train, eval_set=(X_val, y_val), early_stopping_rounds=50)
accuracy = cat_model.score(X_val, y_val)
print("Accuracy: %.2f%%" % (accuracy * 100.0))

0:	learn: 0.5416929	test: 0.5635412	best: 0.5635412 (0)	total: 2.68ms	remaining: 1.45s
1:	learn: 0.4664725	test: 0.4889991	best: 0.4889991 (1)	total: 4.92ms	remaining: 1.33s
2:	learn: 0.4138830	test: 0.4341598	best: 0.4341598 (2)	total: 6.82ms	remaining: 1.22s
3:	learn: 0.3805415	test: 0.4174725	best: 0.4174725 (3)	total: 8.7ms	remaining: 1.17s
4:	learn: 0.3527256	test: 0.3942016	best: 0.3942016 (4)	total: 10.4ms	remaining: 1.12s
5:	learn: 0.3375079	test: 0.3825182	best: 0.3825182 (5)	total: 12.3ms	remaining: 1.1s
6:	learn: 0.3239917	test: 0.3727730	best: 0.3727730 (6)	total: 14ms	remaining: 1.07s
7:	learn: 0.3090762	test: 0.3667810	best: 0.3667810 (7)	total: 16ms	remaining: 1.07s
8:	learn: 0.2970254	test: 0.3615358	best: 0.3615358 (8)	total: 17.7ms	remaining: 1.05s
9:	learn: 0.2868061	test: 0.3541081	best: 0.3541081 (9)	total: 19.4ms	remaining: 1.03s
10:	learn: 0.2799478	test: 0.3510609	best: 0.3510609 (10)	total: 21.2ms	remaining: 1.02s
11:	learn: 0.2706260	test: 0.3483959	best: 0.3483959 (11)	total: 22.8ms	remaining: 1s
12:	learn: 0.2661453	test: 0.3462456	best: 0.3462456 (12)	total: 24.4ms	remaining: 993ms
13:	learn: 0.2605051	test: 0.3426619	best: 0.3426619 (13)	total: 25.9ms	remaining: 978ms
14:	learn: 0.2549583	test: 0.3363530	best: 0.3363530 (14)	total: 27.7ms	remaining: 972ms
15:	learn: 0.2497569	test: 0.3320355	best: 0.3320355 (15)	total: 29.2ms	remaining: 959ms
16:	learn: 0.2441153	test: 0.3298400	best: 0.3298400 (16)	total: 30.7ms	remaining: 947ms
17:	learn: 0.2409281	test: 0.3262885	best: 0.3262885 (17)	total: 32.4ms	remaining: 943ms
18:	learn: 0.2334771	test: 0.3216092	best: 0.3216092 (18)	total: 33.9ms	remaining: 933ms
19:	learn: 0.2293128	test: 0.3190773	best: 0.3190773 (19)	total: 35.6ms	remaining: 930ms
20:	learn: 0.2254262	test: 0.3169948	best: 0.3169948 (20)	total: 37.1ms	remaining: 921ms
21:	learn: 0.2211648	test: 0.3151711	best: 0.3151711 (21)	total: 38.9ms	remaining: 919ms
22:	learn: 0.2174239	test: 0.3153031	best: 0.3151711 (21)	total: 40.6ms	remaining: 917ms
23:	learn: 0.2132543	test: 0.3140192	best: 0.3140192 (23)	total: 42.2ms	remaining: 911ms
24:	learn: 0.2101384	test: 0.3143859	best: 0.3140192 (23)	total: 44.1ms	remaining: 911ms
25:	learn: 0.2071902	test: 0.3090275	best: 0.3090275 (25)	total: 46.3ms	remaining: 918ms
26:	learn: 0.2029773	test: 0.3084215	best: 0.3084215 (26)	total: 48.1ms	remaining: 918ms
27:	learn: 0.1967922	test: 0.3131916	best: 0.3084215 (26)	total: 49.7ms	remaining: 913ms
28:	learn: 0.1939179	test: 0.3098418	best: 0.3084215 (26)	total: 51.5ms	remaining: 911ms
29:	learn: 0.1924468	test: 0.3123539	best: 0.3084215 (26)	total: 53.1ms	remaining: 906ms
30:	learn: 0.1920412	test: 0.3120894	best: 0.3084215 (26)	total: 54.9ms	remaining: 905ms
31:	learn: 0.1889335	test: 0.3128485	best: 0.3084215 (26)	total: 56.7ms	remaining: 903ms
32:	learn: 0.1855072	test: 0.3114742	best: 0.3084215 (26)	total: 58.6ms	remaining: 903ms
33:	learn: 0.1823854	test: 0.3125166	best: 0.3084215 (26)	total: 60.1ms	remaining: 898ms
34:	learn: 0.1788958	test: 0.3093479	best: 0.3084215 (26)	total: 61.7ms	remaining: 894ms
35:	learn: 0.1749224	test: 0.3107830	best: 0.3084215 (26)	total: 63.3ms	remaining: 890ms
36:	learn: 0.1721868	test: 0.3090899	best: 0.3084215 (26)	total: 64.9ms	remaining: 886ms
37:	learn: 0.1720375	test: 0.3092606	best: 0.3084215 (26)	total: 66.6ms	remaining: 884ms
38:	learn: 0.1713699	test: 0.3089562	best: 0.3084215 (26)	total: 68.2ms	remaining: 879ms
39:	learn: 0.1682653	test: 0.3059228	best: 0.3059228 (39)	total: 69.8ms	remaining: 876ms
40:	learn: 0.1643120	test: 0.3090332	best: 0.3059228 (39)	total: 71.5ms	remaining: 874ms
41:	learn: 0.1641586	test: 0.3092276	best: 0.3059228 (39)	total: 73.6ms	remaining: 876ms
42:	learn: 0.1609137	test: 0.3110994	best: 0.3059228 (39)	total: 75.3ms	remaining: 873ms
43:	learn: 0.1573091	test: 0.3114150	best: 0.3059228 (39)	total: 76.8ms	remaining: 869ms
44:	learn: 0.1571697	test: 0.3113936	best: 0.3059228 (39)	total: 78.4ms	remaining: 866ms
45:	learn: 0.1547339	test: 0.3134907	best: 0.3059228 (39)	total: 79.9ms	remaining: 862ms
46:	learn: 0.1546205	test: 0.3134828	best: 0.3059228 (39)	total: 81.5ms	remaining: 858ms
47:	learn: 0.1526230	test: 0.3161402	best: 0.3059228 (39)	total: 83.1ms	remaining: 855ms
48:	learn: 0.1521564	test: 0.3163583	best: 0.3059228 (39)	total: 84.5ms	remaining: 850ms
49:	learn: 0.1520450	test: 0.3164415	best: 0.3059228 (39)	total: 86ms	remaining: 846ms
50:	learn: 0.1494562	test: 0.3172253	best: 0.3059228 (39)	total: 87.7ms	remaining: 844ms
51:	learn: 0.1483615	test: 0.3181409	best: 0.3059228 (39)	total: 89.3ms	remaining: 841ms
52:	learn: 0.1482301	test: 0.3182532	best: 0.3059228 (39)	total: 90.8ms	remaining: 838ms
53:	learn: 0.1469841	test: 0.3182539	best: 0.3059228 (39)	total: 92.3ms	remaining: 834ms
54:	learn: 0.1429885	test: 0.3155260	best: 0.3059228 (39)	total: 93.9ms	remaining: 831ms
55:	learn: 0.1395244	test: 0.3169990	best: 0.3059228 (39)	total: 95.4ms	remaining: 828ms
56:	learn: 0.1390191	test: 0.3169014	best: 0.3059228 (39)	total: 97.1ms	remaining: 826ms
57:	learn: 0.1382907	test: 0.3180109	best: 0.3059228 (39)	total: 98.8ms	remaining: 825ms
58:	learn: 0.1371635	test: 0.3182552	best: 0.3059228 (39)	total: 101ms	remaining: 823ms
59:	learn: 0.1365927	test: 0.3190689	best: 0.3059228 (39)	total: 102ms	remaining: 823ms
60:	learn: 0.1365149	test: 0.3189560	best: 0.3059228 (39)	total: 104ms	remaining: 822ms
61:	learn: 0.1329079	test: 0.3210751	best: 0.3059228 (39)	total: 106ms	remaining: 823ms
62:	learn: 0.1315968	test: 0.3221377	best: 0.3059228 (39)	total: 108ms	remaining: 824ms
63:	learn: 0.1309848	test: 0.3222394	best: 0.3059228 (39)	total: 110ms	remaining: 823ms
64:	learn: 0.1277752	test: 0.3234421	best: 0.3059228 (39)	total: 112ms	remaining: 822ms
65:	learn: 0.1250558	test: 0.3245861	best: 0.3059228 (39)	total: 114ms	remaining: 820ms
66:	learn: 0.1236897	test: 0.3249892	best: 0.3059228 (39)	total: 115ms	remaining: 818ms
67:	learn: 0.1213160	test: 0.3264500	best: 0.3059228 (39)	total: 117ms	remaining: 818ms
68:	learn: 0.1212723	test: 0.3265059	best: 0.3059228 (39)	total: 119ms	remaining: 817ms
69:	learn: 0.1192795	test: 0.3284897	best: 0.3059228 (39)	total: 121ms	remaining: 815ms
70:	learn: 0.1174189	test: 0.3290550	best: 0.3059228 (39)	total: 122ms	remaining: 812ms
71:	learn: 0.1151241	test: 0.3303168	best: 0.3059228 (39)	total: 124ms	remaining: 810ms
72:	learn: 0.1135542	test: 0.3309889	best: 0.3059228 (39)	total: 126ms	remaining: 808ms
73:	learn: 0.1121437	test: 0.3340138	best: 0.3059228 (39)	total: 127ms	remaining: 806ms
74:	learn: 0.1090557	test: 0.3343125	best: 0.3059228 (39)	total: 129ms	remaining: 805ms
75:	learn: 0.1067336	test: 0.3339308	best: 0.3059228 (39)	total: 131ms	remaining: 805ms
76:	learn: 0.1043013	test: 0.3335574	best: 0.3059228 (39)	total: 133ms	remaining: 804ms
77:	learn: 0.1027382	test: 0.3334646	best: 0.3059228 (39)	total: 135ms	remaining: 804ms
78:	learn: 0.1023474	test: 0.3337899	best: 0.3059228 (39)	total: 137ms	remaining: 801ms
79:	learn: 0.1014353	test: 0.3328400	best: 0.3059228 (39)	total: 139ms	remaining: 800ms
80:	learn: 0.0986249	test: 0.3328455	best: 0.3059228 (39)	total: 140ms	remaining: 799ms
81:	learn: 0.0985447	test: 0.3328425	best: 0.3059228 (39)	total: 142ms	remaining: 798ms
82:	learn: 0.0984268	test: 0.3327751	best: 0.3059228 (39)	total: 144ms	remaining: 796ms
83:	learn: 0.0982480	test: 0.3326588	best: 0.3059228 (39)	total: 146ms	remaining: 795ms
84:	learn: 0.0980714	test: 0.3327144	best: 0.3059228 (39)	total: 147ms	remaining: 792ms
85:	learn: 0.0967578	test: 0.3332326	best: 0.3059228 (39)	total: 149ms	remaining: 790ms
86:	learn: 0.0963742	test: 0.3331412	best: 0.3059228 (39)	total: 151ms	remaining: 788ms
87:	learn: 0.0946368	test: 0.3337064	best: 0.3059228 (39)	total: 152ms	remaining: 785ms
88:	learn: 0.0928662	test: 0.3322225	best: 0.3059228 (39)	total: 154ms	remaining: 782ms
89:	learn: 0.0915664	test: 0.3335496	best: 0.3059228 (39)	total: 155ms	remaining: 779ms
Stopped by overfitting detector  (50 iterations wait)

bestTest = 0.3059228392
bestIteration = 39

Shrink model to first 40 iterations.
Accuracy: 91.30%

FINAL ACCURACY : 91.3%

def getDataFrame2(Age, Sex, ChestPainType, RestingBP, Cholesterol, FastingBS, RestingECG, MaxHR, ExerciseAngina, Oldpeak, ST_Slope):
    # Use one-hot encoding for categorical features
    chest_pain_types = ['ASY', 'ATA', 'NAP', 'TA']
    resting_ecg_types = ['ST', 'Normal', 'LVH']
    exercise_angina_types = ['N', 'Y']
    st_slope_types = ['Down', 'Flat', 'Up']

    data = {
        'Age': [Age],
        'Sex': [1 if Sex == 'M' else 0],
        'RestingBP': [RestingBP],
        'Cholesterol': [Cholesterol],
        'FastingBS': [FastingBS],
        'MaxHR': [MaxHR],
        'Oldpeak': [Oldpeak],
        
    }

    # Use a dictionary for mapping categorical variables
    categorical_mapping = {
        'ChestPainType': chest_pain_types,
        'RestingECG': resting_ecg_types,
        'ExerciseAngina': exercise_angina_types,
        'ST_Slope': st_slope_types,
    }

    for cat_feature, cat_values in categorical_mapping.items():
        data.update({f'{cat_feature}_{cat_val}': [1 if cat_val == eval(cat_feature) else 0] for cat_val in cat_values})

    df = pd.DataFrame(data)
    return df

Age = 70

#  [ M or F ]
Sex = 'M'

# [ TA or ATA or NAP or ASY ]
ChestPainType = 'ASY'

RestingBP = 140.0

Cholesterol = 289.0

FastingBS = 0

# [ Normal or ST or LVH ]
RestingECG = 'Normal'

MaxHR = 120

# [ N or Y ]
ExerciseAngina = 'N'

Oldpeak = 1.0

# [ Up or Flat or Down ]
ST_Slope = 'Flat'

df = getDataFrame2(Age,Sex, ChestPainType, RestingBP, Cholesterol, FastingBS, RestingECG, MaxHR, ExerciseAngina, Oldpeak, ST_Slope)
df[nonInt8Columns] = scaler.transform(df[nonInt8Columns])
finalPrediction = cat_model.predict(df)
if finalPrediction[0] == 0:
    print('You are not currently at risk of a Heart Attack')
else:
    print('You are currently at risk of a Heart Attack')

You are currently at risk of a Heart Attack