Unit 7 Unit 9

Unit 8 - Gradient Cost Function

This unit focused on understanding gradient descent for optimizing linear regression models. Experimentation with iteration numbers and learning rates provided insights into cost function convergence and model optimization.

Key Learning Outcomes

  • Gradient Descent Dynamics: Observed how increasing iterations reduced cost, improving model accuracy, and how learning rates impacted convergence speed and stability.
  • Applicability and Challenges: Explored hyperparameter tuning for achieving balance between speed and accuracy, and discussed generalization challenges for different datasets.
  • Ethical Considerations: Highlighted the importance of ensuring algorithm transparency and interpretability when applying optimization methods in real-world contexts.

Key Artefacts

  • Gradient Descent Code: Implemented Python code to optimize slope (m) and intercept (b) while tracking cost reduction over iterations.
  • Observational Insights: Adjusted iteration numbers (e.g., 50, 100) and learning rates (e.g., 0.08, 0.1) to demonstrate their influence on convergence and cost minimization.
  • Discussions and Feedback: Team discussions emphasized the importance of tuning parameters for dataset variability and avoiding overfitting in model optimization.

Self-Reflection

  • Strengths: Gained practical understanding of gradient descent mechanics and hyperparameter tuning for optimization.
  • Improvements: Enhanced skills in identifying the impact of hyperparameter values and addressing challenges in optimization for diverse datasets.

Code Showcase

							
# Gradient descent for linear regression
import numpy as np

def gradient_descent(x, y, lr, iterations):
	m, b = 0, 0
	n = len(x)
	for _ in range(iterations):
		y_pred = m * x + b
		cost = (1/n) * sum((y - y_pred)**2)
		dm = -(2/n) * sum(x * (y - y_pred))
		db = -(2/n) * sum(y - y_pred)
		m -= lr * dm
		b -= lr * db
	return m, b, cost

x = np.array([1, 2, 3, 4])
y = np.array([2, 4, 6, 8])
m, b, final_cost = gradient_descent(x, y, lr=0.01, iterations=1000)
print("Slope:", m, "Intercept:", b, "Final Cost:", final_cost)