University of Southern California
CS 567 - Machine Learning

Time & Place : Tuesday (17:00 - 19:50) @HNB 100 (Spring 2003)

[Announcements][Course Description][Class Format][Grading][Contact Instructor/TA][Office Hours]
[Class Schedule/Lecture Notes][Projects Page][Grade Statistics]


  • 04-23-2003: The Projects Web page has been updated with the project presentation order. Please confirm your team's order. Attendance is compulsory and you will be marked on the basis of your questions and comments during the presentations. 


This course is intended to be a broad exposure to the field of machine learning - a topic which, in essence, deals with the issue of designing machines, algorithms and tools which automatically improve with experience. This course is designed to give a graduate-level student a thorough grounding in the methodologies, technologies, mathematics and algorithms currently needed by people who do research in learning with data. The course will initially discuss the fundamental issues in learning theory including topics like optimization, cost function, information gain, regularization, model selection, VC dimension and PAC learnability. Topics from Inductive and Analytical learning theory including Decision trees, Boosting & Bagging, CART and concept learning/hypotheses evaluation will be addressed. Then, we will look at various forms of supervised learning implementations - both linear (Projection methods, LDA, subspace) and non-linear methods (RBFs, Neural Networks, SVMs, LWL) while also addressing issues of subset/feature selection and dimensionality reduction. Topics from unspervised learning like ICA, Factor Analysis and Clustering will be covered in addition to basics of Reinforcement Learning theory .

Skills from this course will be useful for basic and applied research in the field of Computer Science(e.g. algorithms, data mining), Statistics(e.g. optimization theory, regression), Artificial Intelligence(e.g. pattern recognition, data visualization), Information theory, Finance(e.g. time series prediction) and Cognitive Science.


The syllabus is aimed at covering the state of the art developments in the machine learning community. This will be achieved by first identifying and asking the 'right questions' about fundamental issues that are critical in determining the success or failure of a learning system. Then, implementations under different domains, requirements and constraints will be explored.

Fundamental Issues in Learning Theory

Optimization & Cost function - Least Squares, Minimum-Variance Minimum-Bias Estimates,
Lagrange Methods

Information Gain & Function Spaces

VC Dimension & PAC learnability

Regularization - Green function, P-norm regularization, Smootheness criterion,
Bias Variance Tradeoff
Model Selection - MDL, Bayesian Information Criterion, AIC, Cross Validation

Inductive and Analytical Learning

Decision Trees- MARS, Decision tree representation, overfitting

Hypotheses Evaluation

Learning Rule Sets- FOIL

Explanation Based Learning (EBL)- Deductive learning, Inductive Bias and Knowledge Level

Bootstrap methods- Boosting, Bagging etc.

Supervised Learning - Regression and Classification

Linear Methods
Projection & Subspace methods
Linear Discriminant Analysis & Logistic Regression

Non-linear Methods
Basis Expansion - Splines, Waveletes, RKHS, Generalized Additive Models
Kernel Methods - RBFs, Local Weighted Linear Regression, Kernel Estimation
Neural Networks, Support Vector Machines and Flexible Discriminants - Nearest Neighbours
Bayesian Learning - EM Algorithm, Bayes Belief Network, Max. Likelihood

Feature Selection & Dimensionality Reduction- Global vs Local and Projection methods,
Automatic Relevance Detection (ARD)

Genetic Algorithms - Fitness function, Genetic Programming

Unsupervised Learning Clustering, Vector Quantization, SOMs, Principa Curves, ICA,
Projection Pursuit, Factor Analysis and Latent Variables

Reinforcement Learning - Dynamic Programming, Monte Carlo & TD learning

Class Schedule & Lecture Notes (Check out this page)



Teaching Assistant
Prof. Sethu Vijayakumar
Research Assistant Professor
University of Southern California
Hedco Neurosciences Building HNB-103
Los Angeles, CA 90089-2520
phone: (213) 740 1551
Aaron D'Souza
Graduate Research Assistant
University of Southern California
Hedco Neurosciences Building HNB-001
Los Angeles, CA 90089-2520
phone: (213) 821 6370
Rohan Chitradurga
Masters Course Student (EECN),
Department of Electrical Engineering,
University of Southern California

Hedco Neurosciences Building HNB-001
Los Angeles, CA 90089-2520

Class Mailing List

Office Hours

According to email arrangement with instructor or TA.

Class Format

The course consists of lectures with discussions, reading assignments, and homework assignments. In addition, there will be time allocated for review of some of the latest seminal developments ( e.g. papers published at ICML, NIPS) in key areas. There will be a mid-term exam reviewing the basics of the course. At the end of the course, each student will carry out an independent final project in an area subject to the instructor's approval.


  • 3 Homework Assignments, 30%
  • 1 Mid term, 30%
  • 1 Final Project, 40%


Ideally CS561 or CS442, basic knowledge in linear algebra, statistics, calculus, and programming in MATLAB/C (or another language), or permission by instructor. Working knowledge of probability theory and algorithms is recommended but not required.

Academic Integrity

All students are required to abide by the USC code of Academic Integrity. Violation of that Code will be dealt with as described in SCAMPUS. If you have any questions about the responsibilities of either students, faculty, or graders under this policy, contact the instructor or the Office of Student Conduct.

Disabilities and Academic Accomodation

Students requesting academic accomodations based on a disability are required to register with Disability Services and Programs (DSP) each semester. A letter of verification for approved accomodations can be obtained from DSP when adequate documentaion is filed. Please be sure the letter is delivered to the instructor (or TA) as early in the semester as possible. DSP is open Monday-Friday, 8:30-5:00. The office is in Student Union 301 and their phone number is (213) 740-0776.

If you have comments or suggestions, send email to