ECVision logo Cognitive Vision Model Syllabus




Introduction

This is a syllabus resource for Cognitive Computer Vision, such as might be taught in a comprehensive course on Cognitive Computer Vision. Recognising that what might actually be taught is a subset of this material, we have tried to structure this as a resource, meaning that the given topics are recommended, but the choice of topics for any particular course is up to the lecturer. This is a different resource from the Cognitive Computer Vision Ontology which tries to lay out a view of the structure of Cognitive Computer Vision.

There are many technologies that could have been included, but we are proposing those that we thought had the greatest value for Cognitive Vision systems, and are likely to be the foundation for the summer school course and textbook. This is not a hierarchy, nor are the topics mutually exclusive.

We have tried to identify the central topics here and aimed at a typical full-year course with 54 lecture hours. We think that at a minimum, coverage of each of the five Cognitive Computer Vision subject areas should have an overview, one or more techniques and an example application.

We have tried to be mildly prescriptive about the order of topics, starting with the most important (in our estimation), but are not specifying the method of presentation, nor the depth, all of which will depend on the presenter's preferences and the amount of available time.

Some good general references are:

  1. Forsyth and Ponce. Computer Vision: a modern approach. Prentice-hall, 2002.
  2. Duda, Hart and Stork. Pattern Classification (2nd Edition). Wiley Interscience, 2000.

With ECVision funding, we are still working at: (1) identifying a key citation and (2) collecting online resources for each topic.

Basic prerequisite background knowledge:

Intermediate prerequisite background knowledge:

The Syllabus

There are five components here, and we assume that some material will be taught from each. Each of five components has a minimal time associated with it and also a full time.

  1. Knowledge Representation (3-12 hours)
    1. Overview/Issues (1-2 hours)
      1. Style
        1. Image/Appearance-based
        2. Relational/Graphical
        3. Probabilistic
        4. Ontological
        5. Geometric/Object
        6. Logical/Rule-based/Syntactic
        7. Procedural/Embodied
      2. Issues
        1. Indexing
        2. Certainty
        3. Scale
        4. Multiple Representations
        5. Storage
    2. Knowledge Representation Technologies (2-5 hours)
      1. Receptive Fields/Gaussian Derivatives
      2. Graph Representations
      3. Bayesian Network Models
      4. Hidden Markov Models
      5. Eigenspace / Principal Component Representations
      6. Active/Deformable/Parametric Shape Models
      7. Frames/Rules/Demons
    3. Applications/Case Studies (1-5 hours)
      1. Activity/Behavior/Processes/Dynamics
      2. Classification/Category
      3. Context/Scene/Situations
      4. Function
      5. Objects/Parts
      6. Ontologies
      7. Parameters
      8. Task Control
    4. General Resources
  2. Recognition, Categorization and Estimation (3-14 hours)
    1. Overview/Issues (1-2 hours)
      1. What
        1. Category
        2. Parameters
        3. Position
        4. State
      2. Issues
        1. Accuracy
        2. Genericity
        3. Labeling/Detection/Localization
    2. Recognition Technologies (1-7 hours)
      1. Bayesian Classification
      2. Model Based Indexing, Invocation
      3. Decision Trees, Sequential Classifiers
      4. k-Nearest Neighbor
      5. Neural Network/Perceptron Methods
      6. KMAX
    3. Applications/Case Studies (1-5 hours)
      1. Activity/Behavior/Processes/Dynamics
      2. Classification/Category
      3. Context/Scene/Situations
      4. Function
      5. Objects/Parts
      6. Parameters
    4. General Resources
  3. Reasoning about Structures and Events (4-11 hours)
    1. Overview/Issues (1-2 hours)
      1. Content
        1. Objects & spatial structures and their organisation
        2. Appearance/Visibility
        3. Events & temporal structures and their organisation
        4. Tasks/Goals
      2. Issues
        1. Performance
        2. Prediction
        3. Planning
        4. Decision making
        5. Information fusion
        6. Self-analysis
        7. Uncertainty
    2. Reasoning Technologies (overview) (3-9 hours)
      1. Bayesian Inference
      2. Change and Moving Object Detection
      3. Temporal Event Analysis
      4. Perceptual Organization, Grouping / Figure-Ground Separation
      5. Performance Analysis for Vision
      6. Correspondence Matching
      7. Optimization
      8. Planning for sensing and other processes
      9. Occlusion Understanding and Recovery
      10. Decision Making
        1. Probabilistic
        2. Rule Based
        3. Soft Control
    3. Applications/Case Studies (1-5 hours)
      1. Activity/Behavior/Processes/Dynamics
      2. Classification/Category
      3. Context/Scene/Situations
      4. Function
      5. Objects/Parts
    4. General Resources
  4. Model Learning (2-12 hours)
    1. Overview/Issues (1-2 hours)
      1. Types of Learning
        1. Supervised
        2. Case-based
        3. Process identification: ARMA,ANOVA,HMM
        4. Unsupervised
      2. Issues
        1. Feature Selection
        2. Validation
        3. Learning Control (Robustness, Speed, Presentations, ...)
    2. Learning Technologies (1-5 hours)
      1. Bayesian / Probabilistic Model Learning
        1. Process Identification
        2. Graphical Models
        3. EM
      2. k-Means
      3. Principal Component Approaches
      4. Support Vector Machines
      5. Structure/Rule Learning
    3. Applications/Case Studies (1-5 hours)
      1. Activity/Behaviors/Processes/Dynamics
      2. Classification/Categor
      3. Context/Scenes/Situations
      4. Function
      5. Objects/Parts
      6. Parameters
      7. Task Control
    4. General Resources
  5. Visual Process Control (1-5 hours)
    1. Overview/Issues (1-2 hours)
      1. Issues
        1. Quality/Accuracy
        2. Goal Specification
        3. Multiple/Single Sensor
        4. Distribution of Control
        5. Speed of Response
      2. What is controlled
        1. Sensing
        2. Attention/Focus of processing
        3. Processing Resources
        4. Reasoning Directions
      3. Classes of Control for Vision Systems
        1. Continuous Process Systems
        2. Single Image Processes
        3. Video-rate Systems
    2. Process Control Technologies (1-3 hours)
      1. "Expert-System" Control, Knowledge-Based Systems
      2. Behavior-Based/Reactive Control
      3. Hierarchical Control
      4. Heterarchical/Mixed Control
    3. General Resources
  6. Good example areas and Case Studies (See also VAP book)
    1. Static Image Understanding
      1. Aerial Image Understanding
      2. Scene Understanding
    2. Image Sequence Understanding
      1. Behavior Analysis
      2. Movement Analysis
      3. Walker Identification
      4. Gesture Analysis
      5. Abnormal behavior detection
      6. Expression Understanding