GeoSign

A Web Application for Geolocalized
Signs in Synthesized Swiss German Sign
Language (GeoSign)
Anna Jancso, Xi Rao, Johannes Graën, Sarah Ebling
Institute of Computational Linguistics, University of Zurich
July 14 - ICCHP 2016 - Linz
A Web Application for Geolocalized
Signs in Synthesized Swiss German Sign
Language (GeoSign)
Anna Jancso, Xi Rao, Johannes Graën, Sarah Ebling
Institute of Computational Linguistics, University of Zurich
July 14 - ICCHP 2016 - Linz
DSGS (Deutschschweizerische Gebärdensprache)
➔
GeoSign
Swiss German Sign Language
3
DSGS (Deutschschweizerische Gebärdensprache)
➔
Swiss German Sign Language
➔
Sign language in the German-speaking part of Switzerland
GeoSign
4
Source: https://wiki.erepublik.com/index.php/Deutschschweiz
DSGS (Deutschschweizerische Gebärdensprache)
➔
Swiss German Sign Language
➔
Sign language in the German-speaking part of Switzerland
➔
Around 7,500 users + 13,000 learners
➔
5 dialects: Zurich, Berne, Basel, Lucerne, St. Gallen
GeoSign
5
https://wiki.erepublik.com/index.php/Deutschschweiz
DSGS (Deutschschweizerische Gebärdensprache)
➔
Swiss German Sign Language
➔
Sign language in the German-speaking part of Switzerland
➔
Around 7,500 users + 13,000 learners
➔
5 dialects: Zurich, Berne, Basel, Lucerne, St. Gallen
GeoSign
6
https://wiki.erepublik.com/index.php/Deutschschweiz
DSGS place names
1,829 Swiss station names
GeoSign
7
DSGS place names
1,829 Swiss station names
232 conventionalized signs
GeoSign
8
DSGS place names
1,829 Swiss station names
232 conventionalized signs
GeoSign
1,597 fingerspelled signs
9
DSGS place names
1,892 Swiss station names
232 conventionalized signs
1,597 fingerspelled signs
31 finger alphabet
GeoSign
10
DSGS place names
1,829 Swiss station names
232 conventionalized signs
Example:
L-A-U-T-E-R-B-R-U-NN-E-N
GeoSign
1,597 fingerspelled signs
31 finger alphabet
25 double letters
11
DSGS place names
1,829 Swiss station names
232 conventionalized signs
1,597 fingerspelled signs
56 partial signs
31 finger alphabet
GeoSign
25 double letters
12
Goal
1.
Accessibility of place name signs
2.
Geolocalized form ( → map)
3.
Synthesized signing ( → avatar)
GeoSign
GeoSign Demo
13
Pipeline
resource
collection
GeoSign
database
14
Pipeline
back end
WSGI script
resource
collection
GeoSign
front end
GeoSign website
database
15
Pipeline
back end
WSGI script
resource
collection
GeoSign
front end
GeoSign website
database
16
Resource collection
Swiss stations
→ 1,829
→ German, French, Italian
→ (longitude, latitude)
→ 5 categories: passenger numbers
(high → low)
GeoSign
17
Resource collection
SiGML
→ 232 lexicalized signs
→ 56 partial signs
(31 finger alphabet + 25 double letters)
GeoSign
18
Resource collection
SiGML
SiGML
SiGML
→ 232 lexicalized signs
→ JASigning
232
lexicalized
56 partial
signs signs
(31 finger
→
alphabet incl.)
GeoSign
19
Resource collection
Swiss stations
→ 1,829
→ German, French, Italian
→ (longitude, latitude)
→ 5 categories: passenger numbers
(high → low)
GeoSign
SiGML
→ 232 lexicalized signs
→ 56 partial signs
(31 finger alphabet + 25 double letters)
→ JASigning
20
Database
sigml
GeoSign
(0, *)
string2sigml
(0, *)
letters
21
Example:
LUZERN
Database
SiGML
code
SiGML
code
name
name
freq
(0, *)
sigml
longitude
string2sigml
(0, *)
letters
latitude
lexical sign (T/F)
GeoSign
22
Example:
L-A-U-T-E-R-B-R-U-NN-E-N
Database
SiGML
code
SiGML
code
name
name
freq
(0, *)
sigml
longitude
string2sigml
(0, *)
letters
latitude
lexical sign (T/F)
GeoSign
23
Front end: website
❏
Google Maps
GeoSign
24
Front end: website
❏
❏
Google Maps
Markers: train stations
GeoSign
25
Front end: website
❏
❏
Google Maps
Markers: train stations
❏ Goal:
→ even distribution
→ avoid overlappings
GeoSign
http://pixers.nl/fotobehang/duim-omhoog-midden-33967193
26
Front end: website
❏
❏
Google Maps
Markers: train stations
❏ Goal:
→ even distribution
→ avoid overlappings
❏
Criteria:
→ importance
GeoSign
http://pixers.nl/fotobehang/duim-omhoog-midden-33967193
27
Front end: website
❏
❏
Google Maps
Markers: train stations
❏ Goal:
→ even distribution
→ avoid overlappings
❏
Criteria:
→ importance
→ distance
GeoSign
http://pixers.nl/fotobehang/duim-omhoog-midden-33967193
28
Front end: website
❏
❏
❏
Google Maps
Markers: train stations
❏ Goal: even distribution, no overlappings
❏ Criteria: importance, distance
Search mask
GeoSign
29
Front end: website
❏
❏
❏
❏
Google Maps
Markers: train stations
❏ Goal: even distribution, no overlappings
❏ Criteria: importance, distance
Search mask
Java applet
GeoSign
30
Back end: WSGI script
❏ Processes requests from client
click
marker
SiGML
request
search
user / client
GeoSign
search mask
WSGI script
31
Back end: WSGI script
❏ Retrieves data from database
fetch SiGML
return SiGML
WSGI script
GeoSign
database
32
Back end: WSGI script
❏ Sends requested data back to client
sign
user / client
GeoSign
return SiGML
avatar
WSGI script
33
Summary
Visualization-based tool
→ deaf / hearing sign language
users / learners
→ researchers / interpreters
34
Summary
Visualization-based tool
Sign language synthesis
→ deaf / hearing sign language
users / learners
→ researchers / interpreters
→ flexible/adaptable
(e.g. language change)
35
Summary
Visualization-based tool
Sign language synthesis
Dynamic web app
→ flexible/adaptable
→ prototype
(e.g. language change)
→ future work: other DSGS
dialects / sign languages /
place names
→ deaf / hearing sign language
users / learners
→ researchers / interpreters
36
Thank you!
dankä!
{anna.jancso, xi.rao}@uzh.ch, {graen, ebling}@cl.uzh.ch
GeoSign
37