507: Automatic Inference of Humpback Whalesong Grammar
Proposer: Ashley Walker and Robert Fisher, 650-3098, rbf@aifh
Suggested Supervisors: Fisher, Hallam, Walker
Principal goal of the project:
Previous work in this department has resulted in a self-organizing
network capable of robustly inferring phonemes (or 'whalesong units')
from unsegmented acoustic signals. The primary goal of the present
project is to develop a methodology and programme for analyzing the
structural relationships amongst units classified by this network. If
successful, the programme will be used to determine whether the
structure of songs in our database conform to the currently
accepted grammar. Time permitting, the student will use the programme
to address fundamental research questions including:
- Song Grammar. How well do individual songs conform to the
accepted whalesong grammar proposed by Payne and McVay 
and are there meaningful variations within the songs of an individual
and a population?
- Inter-population song comparison. What significant
differences in song phonology (i.e., number and types of whalesong units)
and arrangement of song sub-sequences (e.g., the length,
complexity, and ordering of sequences of units) exist between the
characteristic songs of different populations?
- Song evolution. What are the mechanisms underlying song
evolution within populations? Do rhythmic changes occur independently
of linguistic drift (unit alterations) and unit sequencing changes?
Humpback whales ( megaptera novaeangliae) emit long,
complex patterned vocalisations, or "songs". A number of discrete
populations of humpback whales exist which, at any point in time, can
be characterised by a unique song shared by all singing
population members. The songs of the various populations which have been
studied all appear to have in common a complex hierarchical structure.
Over a series of years, the characteristic song of each humpback
population changes extensively and irreversibly (within the confines of this
grammar) and, like the songs themselves, song evolution appears to be
governed by syntax-like rules.
While humpback whale songs demonstrate a remarkable amount of regular
high-level structure, they are composed of a variety of complex and
transient elemental phonological "units". Reliable analysis of song
structure requires robust unit classification -- a feature which has
made this process difficult to automate. Recent research in this
department on humpback whale phonology [Mitsakakis 1996; Mitsakakis
et. al 1996; Walker et. al 1997] has demonstrated that it is possible
to reliably detect and classify units using a hierarchical
classification space generated by a topographic mapping algorithm. The
success of this approach arises in part because the complete
acoustic structure of song units is analyzed, rather than the extracted
fundamental pitch frequencies used in previous studies. This technique
is robust to random variations in sound due to the variability
inherent in the marine mammal sound production mechanism and,
consequently, is also robust to the low signal-to-noise conditions of
many hydrophone recordings.
During work in developing the unit classification
algorithm, the structure of songs in our database were observed to
vary in characteristic ways from the hypothesised whalesong
grammar. To closely quantify and assess the significance of these
variations, much larger sets of recordings must be processed. As the
world-wide corpus of recorded whalesong is on the order of several
hundreds of hours large, an automated tool is clearly required.
The present project proposes to create that tool. Specifically, the
student will expand and use the previously developed sub-symbolic unit
detector and classifier as the first processing layer in a
hierarchical programme -- the second layer of which will infer song
structure from the transitions between classified units. It remains to
be determined (by the student in collaboration with the supervisors)
what form this second layer programme should take (e.g.,
string matching algorithm or a connectionist network) and where (if at
all) the boundary between the symbolic and sub-symbolic layers should
The following three further factors (one resulting from the nature
of the humpback whales songs themselves, and two from their
collection) must be taken into account in the construction of the
structural inference layer:
- Recorder fragmenting. Because whalesongs are recorded in the
whale's environment (rather than in conditions better suited to the
recordist) rarely is it possible to track and record a single singer from
the beginning to the end of his song. Therefore, algorithms
which seek to infer song structure from transitions between units must be
capable of operating on song fragments -- i.e., be insensitive to
absolute song starting points.
- Redundant repetitions. The signals themselves are
complicated by the fact that the duration of songs varies -- due to
stutter-like repetitions of units. This phenomena is currently
regarded to be behaviourally insignificant and, therefore, it is
desirable that multiple
occurrences of sound material repeated in this way are ignored.
- Rhythm. The information contained in a whalesong
exists in both the types and ordering of units, as well as the timing of
their delivery. Therefore, whalesong grammar inference algorithms must take
into account where in a song's rhythmic structure particular strings of
This project should lead to a publishable article.
Bob Fisher will be on sabbatical during the MSc project period and so:
1) he will only select a small number of projects to supervise and 2) he will
be in Sweden from June to mid Aug, so 2nd academic supervisor will be involved.
C/C++, MATLAB (Signal processing toolboxes)
Degree of Difficulty:
Medium conceptual difficulty. Full research project is hard and
unclear if convincing results can be obtained, given the previously
unexplored nature of the problem.
Some knowledge of speech or signal processing and connectionist
computing. A strong interdisciplinary interests in acoustic
communication and sensory ecology. Eagerness to explore basic research
Degree Programmes Suitable:
- "Sound of the humpback whale", Payne and MacVay, Science, 1971.
- "Classification of humpback whalesong units using a self
organizing feature map", N. Mitsakakis, MSc Thesis, Department of
Artificial Intelligence, University of Edinburgh, 1996
- "Classification of humpback whalesong units using a self
organizing feature map", N. Mitsakakis, R. Fisher and A. Walker,
J. Acoust. Soc. Am., 100(2), 1996.
- "Singing Maps", A. Walker, R. Fisher and N. Mitsakakis, submitted
to J. Acoust. Soc. Am., 1997.
"A QUANTITATIVE TECHNIQUE TO COMPARE AND CLASSIFY HUMPBACK WHALE
ETHOLOGY, 1988, Vol.77, No.2, pp.89-102
"NORTH PACIFIC HUMPBACK WHALE SONGS - A COMPARISON OF SOUTHEAST
ALASKAN FEEDING GROUND SONGS WITH HAWAIIAN WINTERING GROUND SONGS",
MCSWEENEY_DJ, CHU_KC, DOLPHIN_WF, GUINEE_LN,
MARINE MAMMAL SCIENCE, 1989, Vol.5, No.2, pp.139-148
"BEHAVIORAL CORRELATIONS WITH ABERRANT PATTERNS IN HUMPBACK WHALE
BEHAVIORAL ECOLOGY AND SOCIOBIOLOGY, 1986, Vol.19, No.5, pp.309-312
"HUMPBACK WHALE SONGS AS INDICATORS OF MIGRATION ROUTES",
MARINE MAMMAL SCIENCE, 1990, Vol.6, No.2, pp.155-160
"THE RELATIONSHIP OF SOCIAL VOCALIZATIONS TO SURFACE BEHAVIOR AND
AGGRESSION IN THE HAWAIIAN HUMPBACK WHALE (MEGAPTERA-NOVAEANGLIAE)",
CANADIAN JOURNAL OF ZOOLOGY-JOURNAL CANADIEN DE ZOOLOGIE, 1986,
Vol.64, No.10, pp.2075-2080
"HUMPBACK WHALE SONGS ON A NORTH-ATLANTIC FEEDING GROUND",
MATTILA_DK, GUINEE_LN, MAYO_CA,
JOURNAL OF MAMMALOGY, 1987, Vol.68, No.4, pp.880-883
"UNDERWATER AUDIOGRAM OF A FALSE KILLER WHALE (PSEUDORCA-CRASSIDENS)",
THOMAS_J, CHUN_N, AU_W, PUGH_K,
JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA, 1988, Vol.84, No.3,
"SOUNDS OF A PYGMY RIGHT WHALE (CAPEREA-MARGINATA)",
MARINE MAMMAL SCIENCE, 1992, Vol.8, No.3, pp.213-219
"TIME AND FREQUENCY-DOMAIN CHARACTERISTICS OF SPERM WHALE CLICKS",
JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA, 1995, Vol.98, No.3, pp.1279-1291
Fichtelius and Sjolander, Man's place: Intelligence in Whales, Dolphins and Humans, London:Gollancz, 1973.
"Communication and Behavior in Whales", Washington DC, American Association for
the Advancement of Science, Selected Symposium, 1983
T. Kohonen, "The neural phonetic typewriter", IEEE Computer, pp11-22,
L. Leinonen, R. Mujunen, J. Kangas, K. Torkkola,
"Acoustic pattern of fricative-vowel coarticulation by the self-organising map."
Folia Phoniatrica, 45:173-181, 1993.
I. Nagayama, N. Akamatsu, T.Yoshino.
"Phonetic visualisation for speech training by using neural network."
Proc. Int. Conf. on Spoken Language Processing (ICSLP94), pp 2027-2030, 1994.
V. Rodellar, V. Nieto, P. Gomez, D. Martinez, M. Perez.
"A Neural network for phonetically decoding the speech trace."
Proc. Int. Conf. on Spoken Language Processing (ICSLP94), pp 1575-1578, 1994.
A. I. Hatzis,
"VAHISOM: Visualisation of Articulation for the Hearing Impaired with
Self Organising Maps",
MSc Dissertation, Dept. of Artificial Intelligence, Univ. of Edinburgh, 1995.
"LOCALIZED MEASUREMENT OF EMERGENT IMAGE FREQUENCIES BY GABOR WAVELETS",
BOVIK_AC, GOPAL_N, EMMOTH_T, RESTREPO_A.,
IEEE TRANSACTIONS ON INFORMATION THEORY, 1992, Vol.38, No.2 Pt2,
"MASKED TONAL HEARING THRESHOLDS IN THE BELUGA WHALE",
JOHNSON_CS, MCMANUS_MW, SKAAR_D,
JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA, 1989, Vol.85, No.6,
"VOCALIZATIONS OF THE NORTH-ATLANTIC PILOT WHALE (GLOBICEPHALA-MELAS)
AS RELATED TO BEHAVIORAL CONTEXTS",
BEHAVIORAL ECOLOGY AND SOCIOBIOLOGY, 1990, Vol.26, No.6, pp.399-402