Skip to main content
xplainabledocs
Version: v1.4.1

Rapid Refitting

Lightning-Fast Model Updates

Rapid refitting allows you to update model parameters in milliseconds without retraining from scratch. Perfect for real-time optimization, A/B testing, and parameter tuning scenarios.

Overview​

Rapid refitting is a unique feature of xplainable models that enables instant parameter updates without full retraining. Unlike traditional machine learning models that require complete retraining when parameters change, xplainable models can adjust their activation functions and decision boundaries in real-time.

Key Benefits​

  • Instant updates: Update model parameters in milliseconds, not minutes or hours
  • Real-time optimization: Continuously optimize model performance as new data arrives
  • A/B testing: Test different parameter configurations instantly in production
  • Interactive tuning: Experiment with parameters and see results immediately

How Rapid Refitting Works​

Rapid refitting works by separating the tree structure (which requires full training) from the activation function (which can be updated instantly):

  1. Tree Structure: The decision tree ensemble is built once during initial training with fit()
  2. Activation Function: Parameters like weight, power_degree, and sigmoid_exponent control how leaf values are combined
  3. Instant Updates: Calling update_feature_params() recalculates predictions without rebuilding trees
Understanding the Architecture
  • Tree Building: Computationally expensive, done once during fit()
  • Activation Function: Lightweight mathematical transformation applied to leaf values
  • Rapid Refitting: Updates only the activation function, keeping trees intact

Basic Usage​

Simple Parameter Update​

1from xplainable.core.models import XClassifier
2import pandas as pd
3from sklearn.model_selection import train_test_split
4
5# Load and prepare data
6data = pd.read_csv('data.csv')
7X = data.drop('target', axis=1)
8y = data['target']
9X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
10
11# Train initial model
12model = XClassifier(
13 max_depth=5,
14 min_info_gain=0.01,
15 weight=0.5,
16 power_degree=1,
17 sigmoid_exponent=1
18)
19model.fit(X_train, y_train)
20
21# Initial performance
22initial_eval = model.evaluate(X_test, y_test)
23print(f"Initial ROC AUC: {initial_eval['roc_auc']:.3f}")
24
25# Rapid refit with new parameters across ALL features
26model.update_feature_params(
27 features=model.columns,
28 weight=0.8,
29 power_degree=2,
30 sigmoid_exponent=1.5
31)
32
33# New performance (calculated instantly)
34new_eval = model.evaluate(X_test, y_test)
35print(f"New ROC AUC: {new_eval['roc_auc']:.3f}")
36

The update_feature_params() Method​

This is the core method for rapid refitting. It updates the activation function parameters for a specified subset of features.

1model.update_feature_params(
2 features=['feature_a', 'feature_b'], # List of feature names to update
3 max_depth=6, # Maximum tree depth
4 min_info_gain=0.01, # Minimum information gain for splits
5 min_leaf_size=0.005, # Minimum leaf size
6 ignore_nan=False, # Whether to ignore NaN values
7 weight=0.8, # Activation function weight
8 power_degree=2, # Activation function power degree
9 sigmoid_exponent=1.5, # Activation function sigmoid exponent
10 tail_sensitivity=1.0 # Weight for divisive leaf nodes
11)
12

All parameters are optional -- only the ones you specify will be updated. The features parameter accepts a list of column names. Use model.columns to update all features at once.

Feature-Specific Tuning​

One of the most powerful aspects of rapid refitting is the ability to tune different features with different parameters:

1# Train initial model
2model = XClassifier(max_depth=6, min_info_gain=0.01)
3model.fit(X_train, y_train)
4
5# Fine-tune specific features that are underfitting
6model.update_feature_params(
7 features=['age', 'income'],
8 max_depth=8, # Allow more depth for these features
9 weight=0.9
10)
11
12# Simplify features that are overfitting
13model.update_feature_params(
14 features=['zip_code', 'customer_id_hash'],
15 max_depth=3, # Reduce depth for these features
16 min_info_gain=0.05 # Require more information gain
17)
18
19# Evaluate after feature-specific tuning
20eval_result = model.evaluate(X_test, y_test)
21print(f"ROC AUC after tuning: {eval_result['roc_auc']:.3f}")
22

Parameter Exploration​

1# Test different parameter combinations
2parameter_combinations = [
3 {'weight': 0.3, 'power_degree': 1, 'sigmoid_exponent': 1},
4 {'weight': 0.5, 'power_degree': 1.5, 'sigmoid_exponent': 1},
5 {'weight': 0.7, 'power_degree': 2, 'sigmoid_exponent': 1.2},
6 {'weight': 0.9, 'power_degree': 2.5, 'sigmoid_exponent': 1.5},
7]
8
9results = []
10for i, params in enumerate(parameter_combinations):
11 # Rapid refit with new parameters
12 model.update_feature_params(features=model.columns, **params)
13
14 # Evaluate performance
15 eval_result = model.evaluate(X_test, y_test)
16 roc_auc = eval_result['roc_auc']
17 results.append({
18 'combination': i + 1,
19 'params': params,
20 'roc_auc': roc_auc
21 })
22
23 print(f"Combination {i+1}: ROC AUC={roc_auc:.3f} - {params}")
24
25# Find best parameters
26best_result = max(results, key=lambda x: x['roc_auc'])
27print(f"\nBest parameters: {best_result['params']}")
28print(f"Best ROC AUC: {best_result['roc_auc']:.3f}")
29
30# Apply best parameters
31model.update_feature_params(features=model.columns, **best_result['params'])
32

Advanced Techniques​

Real-Time Optimization with Scipy​

1import numpy as np
2from scipy.optimize import minimize
3from sklearn.metrics import roc_auc_score
4
5def objective_function(params_array, model, X_val, y_val):
6 """Objective function for optimization."""
7 weight, power_degree, sigmoid_exponent = params_array
8
9 # Rapid refit with new parameters
10 model.update_feature_params(
11 features=model.columns,
12 weight=weight,
13 power_degree=power_degree,
14 sigmoid_exponent=sigmoid_exponent
15 )
16
17 # Return negative ROC AUC (minimize)
18 y_scores = model.predict_score(X_val)
19 return -roc_auc_score(y_val, y_scores)
20
21# Split training data for validation
22X_train_opt, X_val, y_train_opt, y_val = train_test_split(
23 X_train, y_train, test_size=0.2, random_state=42
24)
25
26# Train model on optimization set
27model = XClassifier(max_depth=5, min_info_gain=0.01)
28model.fit(X_train_opt, y_train_opt)
29
30# Optimize parameters using rapid refitting
31initial_params = [0.5, 1.0, 1.0] # weight, power_degree, sigmoid_exponent
32bounds = [(0.1, 1.0), (0.5, 3.0), (0.5, 2.0)]
33
34print("Starting optimization...")
35result = minimize(
36 objective_function,
37 initial_params,
38 args=(model, X_val, y_val),
39 bounds=bounds,
40 method='L-BFGS-B'
41)
42
43# Apply optimized parameters
44optimal_weight, optimal_power, optimal_sigmoid = result.x
45model.update_feature_params(
46 features=model.columns,
47 weight=optimal_weight,
48 power_degree=optimal_power,
49 sigmoid_exponent=optimal_sigmoid
50)
51
52print(f"Optimized parameters:")
53print(f" Weight: {optimal_weight:.3f}")
54print(f" Power degree: {optimal_power:.3f}")
55print(f" Sigmoid exponent: {optimal_sigmoid:.3f}")
56print(f" Best ROC AUC: {-result.fun:.3f}")
57

Bayesian Optimization with XParamOptimiser​

XParamOptimiser uses rapid refitting internally. It trains models once per fold during _instantiate(), then uses update_feature_params() to test different parameter configurations quickly:

1from xplainable.core.optimisation.bayesian import XParamOptimiser
2
3# Create parameter optimizer
4optimizer = XParamOptimiser(
5 metric='roc-auc',
6 n_trials=50,
7 n_folds=5,
8 early_stopping=15,
9 random_state=42
10)
11
12# Optimize parameters
13best_params = optimizer.optimise(X_train, y_train)
14
15# Train model with best parameters
16model = XClassifier(**best_params)
17model.fit(X_train, y_train)
18
19print(f"Best parameters: {best_params}")
20evaluation = model.evaluate(X_test, y_test)
21print(f"Test ROC AUC: {evaluation['roc_auc']:.3f}")
22

Regression-Specific Rapid Refitting​

Basic Regression Example​

1from xplainable.core.models import XRegressor
2from sklearn.metrics import mean_squared_error, r2_score
3
4# Train regression model
5regressor = XRegressor(
6 max_depth=5,
7 min_info_gain=0.01,
8 weight=0.5,
9 power_degree=1,
10 prediction_range=(0, 100) # Tuple, not list
11)
12regressor.fit(X_train, y_train)
13
14# Test different activation parameter combinations
15best_metrics = {'MAE': float('inf')}
16best_params = {}
17
18for weight in [0.3, 0.5, 0.7, 0.9]:
19 for power_degree in [1, 1.5, 2, 2.5]:
20 # Rapid refit
21 regressor.update_feature_params(
22 features=regressor.columns,
23 weight=weight,
24 power_degree=power_degree
25 )
26
27 # Evaluate
28 metrics = regressor.evaluate(X_test, y_test)
29
30 if metrics['MAE'] < best_metrics['MAE']:
31 best_metrics = metrics
32 best_params = {
33 'weight': weight,
34 'power_degree': power_degree
35 }
36
37print(f"Best MAE: {best_metrics['MAE']}")
38print(f"Best R2: {best_metrics['R2 Score']}")
39print(f"Best parameters: {best_params}")
40
41# Apply best parameters
42regressor.update_feature_params(features=regressor.columns, **best_params)
43

Combining with XEvolutionaryNetwork​

Rapid refitting and XEvolutionaryNetwork are complementary. Use rapid refitting for activation function parameters, then XEvolutionaryNetwork for deeper weight optimization:

1from xplainable.core.optimisation.genetic import XEvolutionaryNetwork
2from xplainable.core.optimisation.layers import Tighten, Evolve
3
4# Step 1: Train model
5regressor = XRegressor(max_depth=8, min_info_gain=0.01)
6regressor.fit(X_train, y_train)
7
8# Step 2: Optimize tail sensitivity (uses rapid refitting internally)
9regressor.optimise_tail_sensitivity(X_train, y_train)
10
11# Step 3: Apply XEvolutionaryNetwork for weight optimization
12network = XEvolutionaryNetwork(regressor)
13network.add_layer(Tighten(iterations=200, learning_rate=0.05, early_stopping=20))
14network.add_layer(Evolve(mutations=100, generations=50, early_stopping=10))
15network.fit(X_train, y_train)
16network.optimise()
17
18# Evaluate final model
19final_metrics = regressor.evaluate(X_test, y_test)
20print(f"Final MAE: {final_metrics['MAE']}")
21print(f"Final R2: {final_metrics['R2 Score']}")
22
Important

If a model has been optimized using XEvolutionaryNetwork, calling update_feature_params() will overwrite the optimized leaf weights and the model will need to be re-optimized.

Using Feature Importances​

The feature_importances property (not a method call) returns a dictionary of feature importance values:

1# Get feature importances (this is a property, not a method)
2importances = model.feature_importances
3
4# Display sorted by importance
5for feature, importance in sorted(importances.items(), key=lambda x: x[1], reverse=True):
6 print(f" {feature}: {importance:.4f}")
7

Performance Benchmarking​

Speed Comparison​

1import time
2
3def benchmark_rapid_refitting(model, X_train, y_train, X_test, y_test):
4 """Compare rapid refitting vs full retraining performance."""
5
6 # Full retraining benchmark
7 start_time = time.time()
8 new_model = XClassifier(max_depth=5, min_info_gain=0.01, weight=0.8, power_degree=2.0)
9 new_model.fit(X_train, y_train)
10 full_retrain_time = time.time() - start_time
11 full_retrain_eval = new_model.evaluate(X_test, y_test)
12
13 # Rapid refitting benchmark
14 start_time = time.time()
15 model.update_feature_params(
16 features=model.columns,
17 weight=0.8,
18 power_degree=2.0
19 )
20 rapid_refit_time = time.time() - start_time
21 rapid_refit_eval = model.evaluate(X_test, y_test)
22
23 # Multiple rapid refits to show consistency
24 import numpy as np
25 refit_times = []
26 for _ in range(100):
27 start_time = time.time()
28 model.update_feature_params(
29 features=model.columns,
30 weight=np.random.uniform(0.1, 1.0),
31 power_degree=np.random.uniform(0.5, 3.0)
32 )
33 refit_times.append(time.time() - start_time)
34
35 avg_refit_time = np.mean(refit_times)
36
37 print(f"Performance Benchmarking Results:")
38 print(f"{'='*50}")
39 print(f"Full retraining:")
40 print(f" Time: {full_retrain_time:.3f} seconds")
41 print(f" ROC AUC: {full_retrain_eval['roc_auc']:.3f}")
42 print(f"\nRapid refitting:")
43 print(f" Time: {rapid_refit_time:.6f} seconds")
44 print(f" ROC AUC: {rapid_refit_eval['roc_auc']:.3f}")
45 print(f" Average time (100 refits): {avg_refit_time:.6f} seconds")
46 print(f"\nSpeedup: {full_retrain_time/rapid_refit_time:.0f}x")
47
48# Run benchmark
49benchmark_rapid_refitting(model, X_train, y_train, X_test, y_test)
50

Best Practices​

When to Use Rapid Refitting
  1. Parameter tuning -- Quick exploration of parameter space
  2. Real-time optimization -- Continuous model improvement
  3. A/B testing -- Testing different configurations in production
  4. Interactive analysis -- Immediate feedback during exploration
  5. Feature-level tuning -- Address under/overfitting on specific features
Performance Considerations
  1. Always validate on held-out data -- rapid refitting can overfit just like any tuning method
  2. Use model.columns to update all features -- or pass a subset for targeted tuning
  3. Monitor feature_importances -- check how parameter changes affect feature contributions
  4. Combine with XParamOptimiser -- let Bayesian optimization search the parameter space systematically

Next Steps​

Ready for Advanced Optimization?