The three-year SignON project has now finished. The entire SignON consortium would like to thank everyone for their contributions to our project and our co-creation events. In the video below, you will find some consortium members extending heartfelt THANK YOU signs.
What sets SignON apart is that it is one of the first projects to actively engage in a co-creation process with the deaf and hard-of-hearing community. Drawing on lessons learned from this approach, we have authored a white paper focusing on research in sign language technology. This document provides guidelines for effective collaboration, where deaf communities take the lead in steering the research and development processes. You can read the white paper here: https://signon-project.eu/publications/other-publications/ .
The extensive research conducted within SignON raised a lot of new questions and ideas. Firstly, after SignON, different subgroups will continue their work on developing better models for recognition, synthesis, translation, etc. This will be possible not only for researchers from the SignON consortium, but also for others, as the framework and code will be freely available as open-source software. We will also continue to collaborate through our established network. Secondly, we will continue to look into one of the major issues for AI these days: data. We will collect new and process existing data to provide as many resources as possible to further development. We will continue to organise events to promote the work on sign languages and the idea of sign languages as language and cultural treasures. We have drawn up a two-year plan for the advancement of the SignON application into a viable product.
After three years of hard work behind the scenes on our SignON application and service, we have gained better insights into the technical challenges. Our scientific leader, Dr. Dimitar Shterionov, explains in an interview what these challenges are. Watch the video here: https://signon-project.eu/publications/video-series/
From the beginning, the SignON project focused on two complementary areas (i) the technical side for the development of a mobile application for translation between signed and spoken languages and, of course, the related underlying broad research and (ii) working with the stakeholder communities, in the form of co-creation events. By the end of this project, on the one hand we have established a fruitful co-creation framework that adheres strictly to privacy, ethics and legal considerations as well as having developed numerous models (for sign language recognition, natural language understanding, machine translation, synthesis, etc.), all the while pushing further the state-of-the-art in these fields and pipelines. We have also developed new data and processed existing data, making it more suitable for AI, and developed an application and a framework that use these models. Further information on the results is provided in our public deliverables. On the other hand, we have learned a huge amount. While the translation quality is nowhere near satisfactory, we now understand the factors and reasons behind this and know what the requirements are to move beyond the current state-of-the-art. We also learned how to collaborate effectively with such a diverse team. SignON led to the enlargement of the network of researchers and practitioners involved in SL research.
For three years, consortium members of the SignON project have gained valuable insights through collaboration with deaf and hard-of-hearing individuals and communities. Consequently, Jorn Rijckaert, the coordinator of the SignON communication and dissemination, has initiated the creation of a white paper outlining key Do’s and Don’ts in sign language technology. This serves as a guide for inclusive collaboration among policymakers, researchers, and end users. We are in the final stages of completing this white paper, so be sure to keep an eye on our website as we will publish it soon. In the meantime, the interview with Jorn Rijckaert below provides a summary of the most important takeaways we wish to share for upcoming projects.
On Wednesday 29th November, representatives from two research and innovation projects—EASIER (Intelligent Automatic Sign Language Translation), SignON (Sign Language Translation Mobile Application and Open Communications Framework)—and the LEAD-ME (COST) Action met in Brussels to discuss the progress and results of the initiatives, and sign language research more broadly. This was part of the concertation event, “Outcomes and Outlooks of Research on Sign Language in Europe”, organised with the support of the European Commission and moderated by the European Union of the Deaf, a partner in both projects.
The event brought together researchers and end user community representatives who discussed the future of sign language technology in Europe.
In an opening address Member of European Parliament, Brando Benifei, called for accessibility for all citizens to be entrenched in the way we do things. He also stressed that we need to do more. This was echoed in a keynote presentation delivered by the President of the European Union of the Deaf (EUD), Sofia Isari, who noted that we must work together to ensure that the digital world is an accessible world for all.
The projects were introduced by Dimitar Shterionov (Scientific Coordinator for SignON), Eleni Efthimiou (Scientific Coordinator for EASIER), and Krishna Chandramouli (WG3 Leader of the ‘LEAD-ME’ European Cooperation in Science and Technology (COST) Action). Eleni Efthimiou highlighted that minority languages remain disconnected from technology, and that this has a major impact on inclusion.
A panel discussion moderated by Humberto Insolera (Executive Committee Member of the European Disability Forum) explored the way forward for sign language research in Europe, and included Frankie Picron (SignON, EASIER, EUD), Dimitar Shterionov (SignON), Eleni Efthimiou (EASIER), and Krishna Chandramouli (LEAD-ME). Discussing the co-creation process, Dimitar Shterionov highlighted that communication and trust with the community are the most important things to establish. Expanding on this, Eleni Efthimiou discussed the importance of providing features that the community has asked for, including elements of grammar and expressions of emotion. Krishna Chandramouli described the normalisation of captioning in media, and how we might use this momentum to include synthetically generated sign language. Frankie Picron commented on avatars, stressing that they will not replace humans, but rather offer an alternative option—and as he pointed out, “we need more options”. The panel discussed the different possible use cases for sign language machine translation, and reflected on SignON and EASIER—both projects have demonstrated the possibility of sign language machine translation, and have highlighted the challenges posed by lack of data.
A lunchtime session of poster presentations and demonstrations showcased a range of perspectives on this area of research and, in the afternoon, a series of technical presentations focused on specific aspects of the projects, and shared what the teams have learnt throughout the research process. Eleni Efthimiou discussed sign language processing technologies, emphasising that we need to join forces to ensure sign language technologies keep up with the overall technology landscape.
Dimitar Shterionov explained the sign language machine translation pipeline, and how important it is that we standardise the way we collect and analyse data. Expanding on this point, after the event, Shterionov explained:
“Since the inception of machine translation for spoken languages, language technology has evolved significantly, reaching unprecedented levels with end-to-end neural models trained on huge amounts of data. However, when it comes to signed languages, we need to turn to a more analytical approach, decomposing the translation into smaller, more focused tasks, i.e., recognition, translation, synthesis. But even more so, we need to involve the right people. At this stage, we need researchers, linguists, and developers composed of and/or working with deaf communities.”
Caro Brosens (SignON, VGTC) presented on the challenges with sign language data, and emphasised the importance of investing in high quality data and better technological environments to process this data so that we can use it. Brosens expanded on this by saying,
“The data bottleneck in sign language research is often reduced to ‘there is not enough data’ but the situation is a lot more nuanced than that. Not only is there not enough data, only a small amount of said data is of high quality and/or computer readable. Money should therefore not only be put towards creating new (high quality) data, but into developing the necessary technical environment to properly and efficiently process the data as well, otherwise the data remains unusable.”
Davy Van Landuyt (SignON, EASIER, EUD) presented on user engagement within the research process, and shared recommendations for future work—that sign language research projects must have deaf professionals at the steering wheel; that transparent and trustworthy co-creation and science communication processes, led and carried out by deaf professionals are essential to building a relationship with deaf communities; and that hearing researchers must be aware of their social responsibility.
Carlos Duarte (LEAD-ME) discussed web accessibility, noting that sign language must keep up with upcoming developments resulting from the strong progress being made on Artificial Intelligence (AI) models. Krishna Chandramouli explored the role of AI in digital accessibility as digital content becomes more pervasive—”AI technology offers new opportunities to be leveraged through content transformation methodologies”.
Closing the event, Frankie Picron (SignON, EASIER, and EUD) summarized the presentations and emphasised the importance of diversity and inclusivity in the consortium. The red thread or the central theme of the event was focused on advancing research on sign language technologies in Europe. Key elements included collaboration. Much like the European Union, where progress is contingent upon the collective efforts of all Member States, research consortia also need to involve all stakeholders to foster a way forward in sign language technology research, with the ultimate goal of enhancing the lives of the deaf communities within the European Union.
We are thrilled to announce that SignON has been honored with the Engaged Research of the Year Award at the 2023 Science Foundation Ireland Awards. This prestigious accolade was presented at the Research Summit, organised jointly by Science Foundation Ireland and the Irish Research Council. The SignON project, a pan-European initiative led by Professor Andy Way of the ADAPT Centre at Dublin City University, is at the forefront of engaged research, reflecting our commitment to excellence and community involvement.
Lorraine Leeson (SignON TCD), Lianne Quigley (SignON TCD) and Dave Lewis (Interim Director of the ADAPT Centre) accepted the award on behalf of the SignON Team.
Engaged Research aims to improve, understand, or investigate an issue of public interest or concern including societal challenges and is advanced in collaboration with societal partners and this award recognises the important role of Engaged Research in enabling Science Foundation Ireland-funded research to deliver societal impact for Ireland. Historically, research on sign language technologies has predominantly been conducted by hearing researchers with limited understanding of sign language or the Deaf communities. This often led to outputs that were misaligned with the actual needs and cultural contexts of the DHH communities. This award recognised that the SignON project, in contrast, has placed Deaf experts and community members at the centre of the innovation process, ensuring that the technology developed is both meaningful and useful to its end-users.
The SignON project is a beacon of engaged research, embracing a co-creation process that ensures the technology is usable, useful, and accessible, while being applied in a responsible and ethical manner. The European Union of the Deaf has taken a leadership role in the co-creation work package, exemplifying the project’s commitment to Deaf leadership. The engagement process includes a variety of formats such as surveys, interviews, round-tables, workshops, and art-science projects, ensuring diverse input and perspectives.
Internal processes have been meticulously designed to integrate feedback from the DHH community and experts into the project. This has ensured that the voices of the DHH community are not just heard, but are instrumental in shaping the technology. Sign language linguists play a pivotal role in this process, ensuring that the technology is linguistically and culturally aligned with the needs of the Deaf communities.
We held our final consortium meeting in the University of the Basque Country in Bilbao on the 14th & 15th of November in We spent two days discussing the plans for the finalisation of the project.
SignON had a strong presence at the START event in TCD on the 29th of September 2023. START is part of European Researchers’ Night, a Europe-wide public engagement event, which displays the diversity of research and its impact on citizens’ daily lives in fun, inspiring ways. In 2023, there were more than 50 events across 25 countries around Europe.
This free event, held in Trinity College Dublin, and hosted in partnership with the Royal College of Surgeons Ireland and the ADAPT Centre, the SFI Research Centre for AI-Driven Digital Content Technology, invited visitors to take an up-close look at the fascinating research that is shaping our world, explore solutions to society’s biggest problems, and learn about cutting-edge thinking through debates, interactive workshops, screenings and much more.
For this event a demo application was developed and primarily demonstrated by Ruth Holmes to show real time fingerspelling in ISL to users. Rachel Moiselle, Ellen Rushe, Irene Murtagh and Aoife Brady also assisted in demonstration and SignON’s participation in the event.
We also presented a screening of That is the Question, a short film created as part of SignON with funding from Science Foundation Ireland. Coordinated by Dr Shaun O’Boyle and Dr Elizabeth Mathews in collaboration with Alvean Jones and Lianne Quigley, That is the Question is a Shakespeare performance in Irish Sign Language “for people and machines”. It asks how an artificial intelligence might perceive sign language, and features performances from Lianne Quigley and Alvean Jones. Watch the video here: https://signon-project.eu/publications/video-series/ .
