Aspects of the Grammar and Prosody of Finnish Sign Language (ProGram)

 

Academy Research Fellow’s project

Funded by the Academy of Finland for years 2013-2018


Contact: Tommi Jantunen (tommi.j.jantunen[at]jyu.fi), Sign Language Centre / Department of Language and Communication Studies, PO. Box 35 (F), FI-40014 University of Jyväskylä, Finland

Abstract


To date, only a handful of scientific publications have discussed the syntax of Finnish Sign Language (FinSL), and the lack of syntactic knowledge has a negative effect on areas such as FinSL teaching. To produce basic information on FinSL syntax, this Academy Research Fellow’s project, Aspects of the Grammar and Prosody of FinSL (ProGram), focuses on the following syntactic phenomena: sign order, core argument marking, the structure of complex sentences, co-coordination, blends, the prosodic contour and rhythm of sentences, and the prosodic cohesion of discourse. The investigation of the phenomena is based on the widest collection of monologue and dialogue data on FinSL to date and exploits methodologically computer-vision-based video analysis and motion capture technology. The research lays a foundation for the future compilation of FinSL grammar and corpus, and involves multidimensional cooperation with several parties, both nationally and internationally.


People


  1. •Tommi Jantunen – Outi Pippuri – Tuija Wainio – Saija Kuronen – Anna Puupponen – Sanna Nordlund – Danny De Weerdt – Johanna Mesch – Jorma Laaksonen – Kaius Sinnemäki – Carl Börstell – Birgitta Burger – Emma Allingham – Jarkko Keränen


Collaboration


  1. •Sign Language Unit @ The Finnish Association of the Deaf

  2. •Corpus and SignWiki project @ The Finnish Association of the Deaf & Humak University of Applied Sciences

  3. •Corpus Project of Finland’s Sign Languages (CFINSL) @ University of Jyväskylä, Department of Language and Communication Studies, Sign Language Centre

  4. •CoBaSiL project @ Aalto University School of Science, Department of Information and Computer Science

  5. •MoCap laboratory @ University of Jyväskylä, Department of Music, Art and Culture Studies

  6. •Sign Language Section @ Stockholm University, Department of Linguistics

  7. •NoCats project @ Helsinki University, Department of Modern Languages, General Linguistics

  8. •Kielipankki [The Language Bank of Finland] @ FIN-CLARIN


Interviews online


  1. •Tour de CLARIN: Interview with Tommi Jantunen from Finland @ Clarin.eu. May, 2017.

  2. •Kielipankissa: Tommi Jantunen [”In the Language Bank of Finland: Tommi Jantunen”] @ Helsinki.fi. March, 2017.

  3. •Viittomakieli puhuttelee tutkijaa [”Sign language speaks to the researcher”] @ Tietysti.fi. November, 2014.


Selected handouts


In English


  1. •The kinematics of constructed action in sign language narration: A motion capture study. Talk @ The 51st Annual Meeting of the Societas Linguistica Europaea (SLE 2018), University of Tallinn, Estonia, August 29–September 1, 2018.

  2. •Constructed action types and eye behavior in Finnish Sign Language. Talk @ The 8th Conference of the International Society for Gesture Studies (ISGS 8). Cape Town, South Africa, July 4–8, 2018.

  3. •What does constructed action tell us about the nature of syntax in sign languages? Invited talk @ Stockholm University, January 26, 2017.

  4. •Aspects of the rhythm in FinSL and SSL. Talk @ The 12th conference on Theoretical Issues in Sign Language Research (TISLR 12). Melbourne, Australia, January 4–7, 2016.

  5. •Gesture in (Finnish) Sign Language. Invited talk @ The language-gesture connection seminar. University of Oulu, October 22-24, 2015.


In Finnish


  1. •Videoaineiston ja mittausdatan yhdistäminen – tapaus viittomakieli [Combining video and numerical data in sign language research]. Talk @ The 44th Finnish Conference of Linguistics. University of Jyväskylä, May 20, 2017.

  2. •Viittomakielen korpusaineisto ja sen käyttö [The corpus data on Finnish Sign Language and its use]. Kutsuesitelmä @ Kielipankin 20-vuotisjuhlakiertue: Jyväskylä [Invited talk @ The Language Bank of Finland 20 years anniversary roadshow at Jyväskylä], University of Jyväskylä, November 15, 2016.

  3. •Kurkotellen kohti kielioppia – Miten syntaksia on venytetty suomalaisen viittomakielen tutkimuksessa? [Reaching for grammar – How we have bent the notion of ‘syntax’ in the research into FinSL?] Invited talk @ The 43rd Finnish Conference of Linguistics. University of Oulu, 25-27 May, 2016. [abstract]

  4. •Rinnastuksen prosodiaa suomalaisessa viittomakielessä [Prosody of clausal coordination in FinSL]. Talk @ XXIX Phonetics Symposium. Aalto University, March 20–21, 2015.

  5. •Kielitiedettä ja viittomakieliteknologiaa - Kuinka suomalaisen viittomakielen lauseoppia tutkitaan viittomakielen keskuksessa? [Linguistics and sign language technology - How is FinSL syntax studied at the Sign Language Centre?]. Kutsuesitelmä @ Viittomakielen keskuksen avoimet ovet [Invited talk @ SLC open doors event]. University of Jyväskylä, April 25, 2014.


Data


  1. •University of Jyväskylä, Sign Language Centre: ProGram data. The stories Snowman and Frog, where are you? (2016) [video corpus]. – FIN-CLARIN [referred to on dd.mm.yyyy]. Available in Kielipankki, the Language Bank of Finland, at http://urn.fi/urn:nbn:fi:lb-1001100113005. [Jyväskylän yliopisto, viittomakielenkeskus: ProGram-aineisto, lumiukko- ja sammakkotarinat (2016) [videokorpus]. – FIN-CLARIN [viitattu pp.kk.vvvv]. Saatavissa Kielipankin kautta: http://urn.fi/urn:nbn:fi:lb-1001100113005.]


Publications


Submitted


  1. •Börstell, C.; Jantunen, T.; Kimmelman, V.; de Lint, V.; Mesch, J. & Oomen, M. (submitted). Transitivity prominence within and across modalities. [download the CL2017 workshop abstract and the ASAM2018 workshop abstract]

  2. •Jantunen, T. & Takkinen, R. (submitted). Viittomakielten tutkimus [Sign language research]. In M. Luodonpää-Manni, M. Hamunen, R. Konstenius, M. Miestamo, U. Nikanne & K. Sinnemäki (Eds.), Kielentutkimuksen metodologian käsikirja [The methodological handbook of language research; unofficial translation], in preparation for publication with SKS. January 2018.

  3. •Jantunen, T. (submitted). Mikä "viittomassa" on vikana, miten se korjataan ja mitä tästä kaikesta seuraa viittomakielentutkimukselle? [What’s wrong with the ”sign”, how we can fix it, and what are the consequences of this all to the sign language research)?]. In H. Dufva, T. Kelomäki, T. Nieminen & M. Hamunen (Eds.), Käänteitä ja käsitteitä [Turns and concepts; unofficial translation], in preparation for publication with SKS. April 2016. [download manuscript]


2018


  1. •Takkinen, R.; Wainio, T.; De Weerdt, D. & Jantunen, T. (forthcoming in 2018). Suomalainen viittomakieli [Finnish Sign Language]. In E. Tergujeff & M. Kautonen (Eds.), Suullinen kielitaito: Opi, opeta, arvioi [Oral language skills: Learn, teach, evaluate; unofficial translation], in preparation for publication with Finn Lectura.

  2. •Jantunen, T. (2018). Viittomakielet hybridisysteemeinä: hämärärajaisuus ja epäkonventionaalisuus osana viittomakielten rakennetta [Sign languages as hybrid systems: Gradience and unconventionality as a part of the structure of sign languages]. Puhe ja kieli 38(3), 109–126.

  3. •Jantunen, T. (2018). [Elliptical phenomena in] Finnish Sign Language. In J. van Craenenbroeck & T. Temmerman (Eds.), The Oxford Handbook of Ellipsis, pp. 765–784. Oxford: Oxford University Press.

  4. •Burger, B.; Puupponen, A & Jantunen, T. (2018). Synchronizing eye tracking and optical motion capture: How to bring them together. Journal of Eye Movement Research 11(2).


2017


  1. •Jantunen, T. (2017). Fixed and NOT free: Revisiting the order of the main clausal constituents in Finnish Sign Language from a corpus perspective. SKY Journal of Linguistics 30(2017), 137–149.

  2. •Jantunen, T. (2017). Tiedon saatavuudesta [On access to information]. Jyväskylän yliopisto – Tiedonjyvä verkkolehti. https://www.jyu.fi/tiedonjyva

  3. •Jantunen, T. (2017). Constructed action, the clause and the nature of syntax in Finnish Sign Language. Open Linguistics 3, 65-85.


2016


  1. •Jantunen, T. & Pippuri, O. (2016). Snowfrog loikkasi LATiin – ProGramin tarina-aineisto Kielipankissa [Snowfrog is here – ProGram-data published in the Language Bank of Finland]. Kielisilta 3/2016, pp. 13-16.

  2. •Puupponen, A.; Jantunen, T. & Mesch, J. (2016). The alignment of head nods with syntactic units in Finnish Sign Language and Swedish Sign Language. In Proc. Speech Prosody 2016 [organized in Boston (USA), 31 May-3 June, 2016], pp. 168-172.

  3. •Jantunen, T.; Mesch, J.; Puupponen, A. & Laaksonen, J. (2016). On the rhythm of head movements in Finnish and Swedish Sign Language sentences. In Proc. Speech Prosody 2016 [organized in Boston (USA), 31 May-3 June, 2016], pp. 850-853.

  4. •Jantunen, T.; Pippuri, O.; Wainio, T.; Puupponen, A. & Laaksonen, J. (2016). Annotated video corpus of FinSL with Kinect and computer-vision data. In E. Efthimiou, F. Stavroula-Evita, T. Hanke, J. Hochgesang, J. Kristoffersen & J. Mesch (Eds.), Proceedings of the 7th Workshop on the Representation and Processing of Sign Languages: Corpus Mining [organized as a part  of LREC'16 at Portoroz, Slovenia, May 28, 2016], pp. 93-100. Paris: European Language Resources Association (ELRA).

  5. •Takkinen, R.; Jantunen, T. & Seilola, I. (2016). Typological look at kinship terms, colour terms and numbers in Finnish Sign Language. In U. Zeshan & K. Sagara (Eds.), Semantic Fields in Sign Languages: Colour, Kinship and Quantification, pp. 123-162. Sign Language Typology Series 6. Berlin: Mouton De Gruyter.

  6. •Jantunen, T. (2016). Clausal coordination in Finnish Sign Language. Studies in Language 40(1), 204-234.


2015


  1. •Takkinen, R.; Jantunen, T. & Ahonen, O. (2015). Finnish Sign Language. In J. Bakken Jepsen, G. De Clerck, S. Lutalo Kiingi & W. B. McGregor (Eds.), Sign Languages of the World: A Comparative Handbook, pp. 253-272. De Gruyter Handbook Series. Berlin: Mouton De Gruyter.

  2. •Pippuri, O. (2015). Annotointi ja käännöstyö ProGram-projektissa. Opinnäytetyö, viittomakielentulkin koulutusohjelma, Humanistinen ammattikorkeakoulu. http://urn.fi/URN:NBN:fi:amk-201505127633

  3. •Puupponen, A.; Wainio, T.; Burger, B. & Jantunen, T. (2015). Head movements in Finnish Sign Language on the basis of Motion Capture data: a study of the form and function of nods, nodding, head thrusts, and head pulls. Sign Language & Linguistics 18(1), 41-89.

  4. •Jantunen, T. (2015). Rinnastuksen prosodiaa suomalaisessa viittomakielessä [The prosody of clausal coordination in FinSL]. In M. Lehtinen & U. K. Laine (Eds.), XXIX Fonetiikan päivät, Espoo 20.-21.3.2015, Julkaisut – Papers [Proceedings of the XXIX Phonetics Symposium], pp. 15-21. Aalto-yliopiston julkaisusarja Tiede+teknologia 7/2015. Helsinki: Aalto-yliopisto [Aalto University].

  5. •Jantunen, T. (2015). How long is the sign? Linguistics 53(1), 93-124.


2014


  1. •Luzardo, M.; Viitaniemi, V.; Karppa, M.; Laaksonen, J. & Jantunen, T. (2014). Estimating head pose and state of facial elements for sign language video. In O. Crasborn, E. Efthimiou, E. Fotinea, T. Hanke, J. Hochgesang, J. Kristoffersen & J. Mesch (Eds.), Proceedings of the 6th Workshop on the Representation and Processing of Sign Languages: Beyond the Manual Channel [organized as a part  of LREC'14 at Reykjavik, Iceland, May 31, 2014], pp. 105-112. Paris: European Language Resources Association (ELRA).

  2. •Puupponen, A.; Jantunen, T.; Takkinen, R.; Wainio, T. & Pippuri, O. (2014). Taking non-manuality into account in collecting and analyzing Finnish Sign Language video data. In O. Crasborn, E. Efthimiou, E. Fotinea, T. Hanke, J. Hochgesang, J. Kristoffersen & J. Mesch (Eds.), Proceedings of the 6th Workshop on the Representation and Processing of Sign Languages: Beyond the Manual Channel [organized as a part  of LREC'14 at Reykjavik, Iceland, May 31, 2014], pp. 143-148. Paris: European Language Resources Association (ELRA).

  3. •Karppa, M.; Viitaniemi, V.; Luzardo, M.; Laaksonen, J. & Jantunen, T. (2014). SLMotion - An extensible sign language oriented video analysis tool. In N. Calzolari, K. Choukri, T. Declerck, H. Loftsson, B. Maegaard, J. Mariani, A. Moreno,J. Odijk & S. Piperidis (Eds.), Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14) [organized at Reykjavik, Iceland, 28-30 May, 2014], pp. 1886-1891. Paris: European Language Resources Association (ELRA).

  4. •Viitaniemi, V.; Jantunen, T.; Savolainen, L.; Karppa, M. & Laaksonen, J. (2014). S-pot - A benchmark in spotting signs within continuous signing. In N. Calzolari, K. Choukri, T. Declerck, H. Loftsson, B. Maegaard, J. Mariani, A. Moreno,J. Odijk & S. Piperidis (Eds.), Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14) [organized at Reykjavik, Iceland, 28-30 May, 2014], pp. 1892-1897. Paris: European Language Resources Association (ELRA).


2013


  1. •Jantunen, T. (2013). ProGram: näkökulmia suomalaisen viittomakielen kielioppiin ja prosodiaan [ProGram: Aspects of the grammar and prosody of FinSL]. Kielisilta 4/2013, p. 22.


Prior work


The project builds on the existing work on FinSL syntax and sign language technology, for example:


  1. •Jantunen, T. (2013). Ellipsis in Finnish Sign Language. Nordic Journal of Linguistics 36(3), 303-332.

  2. •Luzardo, M; Karppa, M.; Laaksonen, J.; Jantunen, T. (2013). Head pose estimation for sign language video. In J.-K. Kamarainen & M. Koskela (Eds.), Image Analysis [18th Scandinavian Conference, SCIA 2013, Espoo, Finland, June 17-20, 2013. Proceedings], pp. 349-360. Lecture Notes in Computer Science, Vol. 7944. Springer.

  3. •Jantunen, T.; Burger, B.; De Weerdt, D.; Seilola, I. & Wainio, T. (2012). Experiences from collecting motion capture data on continuous signing. In O. Crasborn, E. Efthimiou, E. Fotinea, T. Hanke, J. Kristoffersen, J. Mesch (Eds.), Proceedings of the 5th Workshop on the Representation and Processing of Sign Languages: Interactions Between Corpus and Lexicon [organized as a part of LREC 2012 at Istanbul, Turkey, 27 May, 2012], pp. 75-82. Paris: ELRA.

  4. •Karppa, M.; Jantunen, T.; Viitaniemi, V.; Laaksonen, J.; Burger, B. & De Weerdt, D. (2012). Comparing computer vision analysis of signed language video with motion capture recordings. In N. Calzolari, K. Choukri, T. Declerck, M. Ugur Dogan, B. Maegaard, J. Mariani, J. Odijk & S. Piperidis (Eds.), Proceedings of the 8th international conference on Language Resources and Evaluation conference (LREC 2012), organized at Istanbul, Turkey, May 21-27, 2012. [online publication, available at http://www.lrec-conf.org/proceedings/lrec2012/index.html; 6 pages]

  5. •Jantunen, T. (2008). Fixed and free: order of the verbal predicate and its core arguments in declarative transitive clauses in Finnish Sign Language. SKY Journal of Linguistics 21(2008), 83-123.

  6. •Jantunen, T. (2007). The equative sentence in Finnish Sign Language. Sign Language & Linguistics 10(2), 113-143.



Last modified January 11, 2019