ML Pro Expert-ML modeling and optimization

Example

Given a messy CSV for credit default, ML Pro Expert inspects missingness, target leakage (e.g., post-event variables), data types, and class imbalance; then proposes a reproducible preprocessing graph. In Python: build `Pipeline([('prep', ColumnTransformer([... numeric: StandardScaler; categorical: OneHotEncoder(handle_unknown='ignore') ])), ('clf', ...)])` with train/validation split respecting temporal order or stratification. In R: use tidymodels’ recipes to `step_impute_median`, `step_dummy`, `step_normalize`, `step_smote` for imbalanced classes. In Weka: chain `ReplaceMissingValues → Normalize → AttributeSelection (InfoGain) → SMOTE`.

Scenario

Real-world onboarding of a bank’s tabular data (100+ features). The assistant flags that 'days_since_last_payment' leaks outcome information for customers who’ve already defaulted, recommends removing/lagging it, and constructs a preprocessing pipeline that is fitted only on training folds to prevent leakage. Deliverables: a summary of data quality issues, a sanitized dataset, and a ready-to-run preprocessing pipeline.

Example

Start with simple, defensible baselines (LogisticRegression/NaiveBayes) and escalate to tree ensembles or neural nets when justified. In Python: `RandomizedSearchCV` over `RandomForestClassifier(n_estimators, max_depth, max_features)` with `StratifiedKFold(n_splits=5, shuffle=True)` and a custom scorer for PR-AUC; optionally switch to LightGBM/XGBoost with categorical handling. In R (tidymodels): define a `workflow()` + `tune_grid()` over `rand_forest(mtry, trees, min_n)` with `vfold_cv(v=5, strata=target)`. In Weka: script the Experimenter to compare `J48`, `RandomForest`, and `CostSensitiveClassifier` with 10-fold CV and report mean/variance of F1.

Scenario

E-commerce return prediction where false negatives are costly. The assistant formalizes the cost matrix, tunes cost-sensitive models, and selects the model maximizing expected utility under business costs. It also produces threshold curves so operations can pick decision thresholds aligned with staffing capacity.

Example

Compute and visualize calibrated probabilities, lift/gain charts, ROC/PR curves, and confusion matrices at chosen thresholds. Provide global and local explanations: permutation importance, partial dependence/ICE, and SHAP-style reasoning (or Weka’s attribute evaluation scores). Generate a concise, stakeholder-friendly report: 'Top 5 drivers of churn, stability across folds, what-if analyses, and known limitations.'

Scenario

Healthcare readmission risk model. The assistant shows that PR-AUC is the more appropriate metric (rare event), demonstrates that calibration improves with isotonic regression, highlights 'prior admissions' and 'medication count' as stable drivers across folds, and provides a safe operating threshold achieving recall ≥ 0.75 while constraining false alerts per 100 patients.