[Announcements][Course Description][Class Format][Grading][Contact
Instructor/TA][Office Hours]
[Class Schedule/Lecture
Notes][Projects
Page][Grade Statistics]
|
|
|
|
[Announcements][Course Description][Class Format][Grading][Contact
Instructor/TA][Office Hours]
|
|
| This course is intended
to be a broad exposure to the field of machine learning - a topic which,
in essence, deals with the issue of designing machines, algorithms
and tools which automatically improve with experience. This course
is designed to give a graduate-level student a thorough grounding in
the methodologies, technologies, mathematics
and algorithms currently needed by people who do research in
learning with data. The course will initially discuss the fundamental
issues in learning theory including topics like optimization, cost function,
information gain, regularization, model selection, VC dimension and
PAC learnability. Topics from Inductive and Analytical learning theory
including Decision trees, Boosting & Bagging, CART and concept
learning/hypotheses evaluation will be addressed. Then, we will look
at various forms of supervised learning implementations - both linear
(Projection methods, LDA, subspace) and non-linear methods (RBFs, Neural
Networks, SVMs, LWL) while also addressing issues of subset/feature
selection and dimensionality reduction. Topics from unspervised learning
like ICA, Factor Analysis and Clustering will be covered in addition to
basics of Reinforcement Learning theory .
Skills from this course will be useful for basic and applied research in the field of Computer Science(e.g. algorithms, data mining), Statistics(e.g. optimization theory, regression), Artificial Intelligence(e.g. pattern recognition, data visualization), Information theory, Finance(e.g. time series prediction) and Cognitive Science. |
|
The syllabus is aimed at covering the state of the art developments
in the machine learning community. This will be achieved by first identifying
and asking the 'right questions' about fundamental issues that
are critical in determining the success or failure of a learning system.
Then, implementations under different domains, requirements
and constraints will be explored. |
|
|
Fundamental
Issues in Learning Theory |
|
|
Optimization &
Cost function - Least Squares, Minimum-Variance Minimum-Bias
Estimates, Lagrange Methods |
|
|
Information Gain
& Function Spaces |
|
|
VC Dimension &
PAC learnability |
|
|
Regularization - Green
function, P-norm regularization, Smootheness criterion, Bias Variance Tradeoff |
|
|
Model Selection - MDL, Bayesian Information Criterion, AIC, Cross
Validation |
|
|
Inductive
and Analytical Learning |
|
|
Decision Trees-
MARS, Decision tree representation, overfitting |
|
|
Hypotheses Evaluation |
|
|
Learning Rule Sets-
FOIL |
|
|
Explanation Based
Learning (EBL)- Deductive learning, Inductive Bias and
Knowledge Level learning |
|
|
Bootstrap methods-
Boosting, Bagging etc. |
|
|
Supervised
Learning - Regression and Classification |
|
|
Linear Methods
Projection & Subspace methods
Linear Discriminant Analysis & Logistic
Regression |
|
|
Non-linear Methods
Basis Expansion - Splines,
Waveletes, RKHS, Generalized Additive Models
Kernel Methods - RBFs, Local Weighted Linear
Regression, Kernel Estimation
Neural Networks, Support Vector Machines
and Flexible Discriminants - Nearest Neighbours
Bayesian Learning - EM Algorithm, Bayes Belief Network, Max. Likelihood |
|
|
Feature Selection
& Dimensionality Reduction- Global vs Local
and Projection methods, Automatic Relevance Detection (ARD) |
|
|
Genetic Algorithms
- Fitness function, Genetic Programming
|
|
|
Unsupervised
Learning Clustering, Vector Quantization, SOMs,
Principa Curves, ICA, Projection Pursuit, Factor Analysis and Latent Variables |
|
|
Reinforcement
Learning - Dynamic Programming,
Monte Carlo & TD learning |
| Instructor |
Teaching Assistant |
Grader |
| Prof. Sethu Vijayakumar
Research Assistant Professor University of Southern California Hedco Neurosciences Building HNB-103 Los Angeles, CA 90089-2520 phone: (213) 740 1551 email: sethu@usc.edu |
Aaron D'Souza Graduate Research Assistant University of Southern California Hedco Neurosciences Building HNB-001 Los Angeles, CA 90089-2520 phone: (213) 821 6370 email: adsouza@usc.edu |
Rohan Chitradurga Masters Course Student (EECN), Department of Electrical Engineering, University of Southern California Hedco Neurosciences Building HNB-001 Los Angeles, CA 90089-2520 email: chitradu@usc.edu |
Class FormatThe course consists of lectures
with discussions, reading assignments, and homework assignments. In addition,
there will be time allocated for review of some of the latest seminal
developments ( e.g. papers published at ICML, NIPS) in key areas. There
will be a mid-term exam reviewing the basics of the course. At the end of
the course, each student will carry out an independent final project in
an area subject to the instructor's approval.
Grading
PrerequisitesIdeally CS561 or CS442, basic knowledge in linear algebra, statistics, calculus, and programming in MATLAB/C (or another language), or permission by instructor. Working knowledge of probability theory and algorithms is recommended but not required.Academic IntegrityAll students are required
to abide by the USC
code of Academic Integrity. Violation of that Code will be dealt
with as described in SCAMPUS. If you have any questions about the responsibilities
of either students, faculty, or graders under this policy, contact
the instructor or the
Office
of Student Conduct.
Disabilities and Academic AccomodationStudents requesting academic
accomodations based on a disability are required to register with Disability
Services and Programs (DSP) each semester. A letter of verification for
approved accomodations can be obtained from DSP when adequate documentaion
is filed. Please be sure the letter is delivered to the instructor (or TA)
as early in the semester as possible. DSP is open Monday-Friday, 8:30-5:00.
The office is in Student Union 301 and their phone number is (213) 740-0776.
|
If you have comments or suggestions, send email to sethu@usc.edu