Discovering criminal and terrorist networks is the primary task of law enforcement and intelligence agencies across Europe. Criminals and terrorists use voice communication over different media. While the personal communication within their networks is usually performed within standard mobile networks and VoIP-type communications (skype, Google Hangouts, others), there might be a significant share of voice in public media, typically hate or propaganda speech on Youtube, Facebook or other social media channels. Determining and tracking target identities across such channels is extremely difficult, and speaker identification (SID) techniques (such as investigated in the European SiiP project) might not be effective in such challenging environments considering isolated data from one speaker only.

From 1 September 2019 , Additess LTD (www.Additess.com) takes part in the consortium (24 members) of the ROXANNE project (http://roxanne-euproject.org/). The project will have a duration of 3 years. This project proposes to combine the strengths of speaker data mining and link analysis to provide LEAs an efficient tool to track and uncover criminals and terrorists. The project will not process speaker data separately, but:

• Make massive use of conversational nature of speech data – in case we know that A speaks often to B, then detecting A on one side of the call will automatically increase the prior probability of B even if the acoustic evidence is not reliable (due for example to illness, channel change or noise). A reliable diarization (determining who spoke when in the conversation) will be developed in the project as a crucial component for this analysis.

• Use of call content. Standard text-independent speaker identification ignores the content of the call, while a simple sentence “Peter speaking” heard on two different calls can completely change the game. It is out of scope of the project to develop perfect speech-to-text (S2T) engines for all possible languages but commercially available ones will be deployed to generate relevant content information and combine it with acoustic speaker information. For languages with missing S2T, language-independent techniques such as universal phoneme sets or automatically determined acoustic units (AUD) will be used.

• Meta-information is crucial for link analysis. Some of it is available (phone and IMEI numbers, geographical information, time-stamps) but the targets are aware such information is collected and have developed ways to falsify or obscure it (one-shot usage of prepaid SIM cards, use of Internet anonymization services, etc). Significant amount of meta-information can however be automatically extracted from the speech signal – for example, automatic detection of age, gender and accent of call participants. For example, identification of a pimp in an illegal child sexual network can be helped by the fact that his calls are predominantly to persons in <20 age category. Another interesting meta-information is the environment – even if the speaker changes his cell phone number every day, he is not likely to change his favorite car. Detecting that a call took place in given car can help the investigation.

• By time-relation analysis, a classical problem of speaker recognition (speaker speaking very little in a call) can be turned into an advantage, as this speaker can simply be identified by the fact that he is speaking little. Hierarchy and trust can be also partially inferred from this analysis.

Data will be crucial for project success. As this project can not count on huge amounts of real investigation or wire-tap data, most of the R&D work will be done on data from public resources: media and social networks. However, we count on exercises performed on real data by LEAs participating in the consortium, that will provide the developers a valuable feedback.

The result of the project will be a prototype of system capable of

• ingesting a significant amount of voice data from different media, along with meta-information,

• analyzing this data in unsupervised or lightly supervised way.

• Presenting the resulting network analysis and converting it to forms integrable with standard investigation SW solutions, such as IBM i2 Analyst Notebook.

For more information regarding the ROXANNE Project, please follow the below:

Website: http://roxanne-euproject.org/

Twitter: https://twitter.com/ROXANNE_project