Will AI take over diplomatic reporting? (WebDebate #56)

Event summary

As a part of a just-in-time experiment, we generated a few human and AI reports from the web debate.

 

'Human report: a summary of web debate by expert

Dr Katharina Hone prepared the following report from the web debate: 

Our first WebDebate in 2023 asked: Will AI take over diplomatic reporting? Four speakers joined us to address this question: Dr Jovan Kurbalija, director of DiploFoundation; Ambassador Umej Bhatia, the head of Singapore’s mission to Geneva and Vienna; Irena Zubcevic, senior advisor to the Permanent Mission of Croatia to the UN in New York; and Liz Galvez, former senior British diplomat and Diplo senior fellow and lecturer. More than 200 participants from around the world joined the debate and contributed to a lively discussion in the chat. 

In the following, we are providing the ‘human’ summary of the WebDebate. Part of the AI-generated summary of the debate was done in real-time and presented by the head of our AI team, Jovan Njegic, in the showcase following the debate. This human summary presents key points from each speaker, the answer to one of the Q&A questions, and a summary of key points.

Setting the scene, Kurbalija described diplomacy as a piece of machinery and placed diplomatic reporting at its core. Historic examples underscoring the centrality of reporting include the Armana archives and Renaissance missions. The first ministry of foreign affairs, established by Cardinal Richelieu in 17th century France, was built around the diplomatic archive. To this day, a considerable amount of diplomats’ time is spent on reporting. Kurbalija argued that some of the reporting work of diplomats cannot be automated but that exploring which aspects can be taken on by AI is crucial for the future of the profession and diplomatic training. 

Jovan Kurbalija: What is part of the diplomatic garden that can be kept from the avalanche of AI?

Ambassador Bhatia highlighted three elements of the work of a multilateral mission: 1) reporting from multilateral meetings; 2) bilateral exchanges and lobbying, and 3) analytical reporting. All three areas of work can be assisted by AI tools. Yet, ‘diplomats in the loop’ will always be required.

First, when it comes to reporting about national statements from multilateral meetings, most of the time established positions are re-stated. In these instances, AI tools can be useful for recording, transcribing, and summarising. This could save smaller missions and international organisations a lot of time. The ‘diplomats in the loop’ are required to check for accuracy and address misunderstandings, mistranslations, possible double-meanings, and the implications of certain phrases.

Second, regarding bilateral work and alliance building, AI tools might be helpful by fine-tuning talking points, acting as an aide memoir, and suggesting alternative formulations or phrasing. Third, in analytical reporting, AI may serve for background research and collecting data. But this area of diplomatic work crucially relies on human judgement and interpretation. 

Ambassador Bhatia: Texts written by AI can mimic authority without any substance. Contextual and situational judgement is crucial and cannot be provided by AI. 

Drawing on her experience as former focal point at the UN Department of Economic and Social Affairs for voluntary national reviews on SDG implementation, Irena Zubcevic highlighted ways in which AI tools can support the SDG review process. Since working in silos is not useful for SDG implementation, AI could help to analyse data, search and combine various data basis, provide trends, and help in understanding the impact of policy actions on different sectors. AI, however, would not be capable, in any meaningful way, of (1) interpret trends; (2) making decisions on how to utilise interlinkages and trade-offs between SDG implementation in different sectors; and (3) correcting the course of SDG implementation. In a sense, AI lacks creative thinking for these kinds of tasks. SDG reporting often faces limited human capacities. Developing countries in particular could benefit from AI tools in support of their SDG review process. User-friendly, and freely or cheaply available AI tools are a requirement for that. 

Irena Zubcevic: Can AI be used for diplomatic reporting? Yes. Can AI do better than humans? It depends. 

Liz Galvez introduced a distinction between factual and analytical reporting. Galvez’s experience reporting from the UN General Assembly general debate is an example of factual reporting. It is mainly done for the records of the ministry to have a synthesis of what was said, when, and by whom. The verbatim record of this already exists, but diplomats produce a more digestible form for the purposes of their own ministry. Doing this can be a tedious job. Speeches are repetitive, overrun, and it is very unlikely that something new will be said that was not signaled in advance.

This type of reporting, at least in part, is an ideal candidate for AI automation. Human diplomats will be needed to provide commentary, context, and judgement on relevancy regarding strategic interests, and highlight unusual or noteworthy incidents. Similarly, standard monthly economic or political roundups, based on media and other publicly available materials, are also candidates for AI-automated diplomatic reporting.

Diplomats would only be left with the task of adding succinct commentary and proving political judgment. AI cannot meaningfully answer questions such as: what should happen next? How is the country’s national interest or policy affected? It also remains unclear how the veracity of sources is ensured and how an AI tool can deal with contradictory public information or opposing views. Diplomats also use their own observations, information from their network of trusted contacts, and off-the-record conversations for their reporting. It is clear that these inputs to reports cannot be replaced by AI. 

Liz Galvez: I can’t see any circumstances where AI would ever be able to produce that kind of analysis because it is based on human interaction and connections and on political judgement. 

Q&A: Could AI help with drafting resolutions?

Speakers agreed that AI will be able to help in producing the zero draft of a resolution, which forms the basis for further negotiation. An AI draft might be good enough to only leave a few words or phrases up for further negotiations. Further, for diplomats stuck in tense and exhausting negotiations, an AI might suggest alternative text formulations or alternative words. Trust is crucial. Parties to the resolution will need re-assurances regarding algorithmic biases. Further, an AI tool might need to be trained not only on previous successful resolutions but also on failed resolutions. AI tools can also help with background research such as finding relevant related resolutions and creating an overview of the previous positions of countries on this or related topics . 

Key points from the discussion

• AI can be useful for certain types of diplomatic reporting and in clearly defined supporting roles. 
• Diplomats always need to remain in the loop to fact-check, deal with ambiguities, or provide analysis and political judgement. 
• Critical analysis and human judgement cannot be replaced by AI and are crucial for most diplomatic reports. 
• Delegating certain reporting tasks to AI will free up time for diplomats to do what AI cannot do. 
• With the introduction of AI tools for reporting, diplomats will have new roles and need new skills. 
• AI tools need to be made available to diplomats from developing countries. 
• There is a potential role for the UN or other international organisations to provide AI reporting and text drafting tools as a service to diplomats.

 

Lateral AI report 

In addition to the narrative report, the Lateral AI report includes a structured taxonomy of debate topics and mentioned names and entities. This report also deduces questions and answers from the debate, which could help develop 'prompting' skills for communication with AI. You can consult the lateral report  here

 

Short AI narrative report

At the recent WebDebate on the role of AI in diplomacy, participants discussed the potential of AI to revolutionize the diplomatic process. Su Sonia Herring argued that AI and human intelligence each have their own place, with human input preferable for non-public and sensitive information and negotiations. Jovan Njegic added that AI should be used with human oversight and that an adequate model must be chosen and improved with reinforcement learning techniques to generate adequate text.

Sorina Teleanu suggested the use of AI to automate the collection and sorting of data to facilitate more precise analysis, while Jovan Kurbalija argued that AI is a necessary component of modern diplomacy and that customized AI is essential for Ministries of Foreign Affairs and other organizations to develop and preserve their own intelligence and way of thinking. Irena Zubcevic discussed the principles guiding Voluntary National Reviews (VNRs) and how AI can be an excellent way to centralize data and policies and reduce the reporting burden. Katarina Bojovic then underlined the importance of taxonomies for AI development and knowledge management.

Ambassador Umej Bhatia asserted that AI can assist with multilateral meeting reports, bilateral exchanges, and analytical reporting, but cannot replace diplomatic writing and analysis. Katharina Hoene concluded by emphasizing the importance of considering the technical, philosophical, and diplomatic perspectives when making use of the potential of AI in diplomacy.

Other participants such as Liz Galvez, Anja Djajic, and Stephanie Borg Psaila discussed the importance of human intervention in diplomatic reporting and how AI can be used to enhance, rather than replace, the work of diplomats. They argued that AI can offer alternative formulations of text, but cannot replace the human touch and trust needed for decision making in diplomatic affairs. AI can be used to some extent to assist with multilateral meeting reports and analytical reporting, but it cannot generate the base zero text. AI may be able to help with the mathematical aspect of reporting, but cannot replace the creative aspect of storytelling.

Overall, all speakers agreed that AI should be used with caution in diplomacy and with human oversight, as it is difficult to verify the accuracy of sources. AI can help free up time for diplomats to focus on their core roles, and can be immensely useful for reporting for countries with limited resources. However, it cannot replicate the human touch and understanding needed for diplomatic affairs.

 

Long AI narrative report 

Liz Galvez, a former multilateral diplomat, gave a speech about the use of Artificial Intelligence in diplomatic reporting. She highlighted the limitation of AI in this area, stating that it is only suitable for ‘chronicling events that are basically factual, publicly accessible information’. She further elaborated that adding a ‘gloss’ to the reporting and ensuring accuracy requires human intervention. 

Galvez underlined that diplomatic reporting should be more than just facts and include insights and judgments. In this regard, she mentioned the story of Sir Kim Darroch and how his diplomatic telegrams assessing the Trump administration were leaked to the British media. She stated: “The most important thing for a government is how does this affect my country's national interest or my country's relationship with that country or my country's policy on this issue, given what the positions of other countries might be. That is the important thing, that interpretive side, I don't think can be done by anything other than a human being”. 

Furthermore, Galvez suggested that AI can be used to search through databases and research relevant resolutions. She noted that some countries might be able to afford their own resources while international organizations could set up such facilities for those who cannot.

Overall, Galvez argued that while AI could be useful in areas such as chronicling publicly accessible information, diplomatic reporting should be left to the judgement of the human eye.:

At a recent Zoom chat, Su Sonia Herring discussed the use of AI and human intelligence in diplomatic work. She said that "each of them have their own place" and that "for non-public and sensitive information and sensitive discussions and negotiations, quite a few of our participants stated that that shouldn't be left to AI". It was also discussed that there is a risk that AI-generated reports may become standardized and repetitive if not monitored properly.

Herring suggested that the UN should mandate its tech envoy to address the use of AI tools in diplomacy and UN work. She also highlighted the potential for AI to be useful for SDG reporting and tracking, as India has already developed a database for this purpose. Moreover, AI can be used to prepare draft resolutions and convention texts as a basis for multilateral negotiations, as nuances can be added in later by humans.

"It was kind of agreed that that would be a good use since they tend to be, they tend to follow a certain patent pattern. And then the most important parts are the nuances, which could be later added by the human perspective," said Herring.:

Ambassador Umej Bhatla discussed the use of AI in diplomatic missions during a panel discussion, noting that while AI is changing the world, it cannot replace high touch diplomacy. He stated that, “High tech cannot necessarily replace high touch. So existing AI tools in the market can't replace high touch, which the diplomats trade and craft.”

In terms of potential uses for AI, he discussed how AI tools such as Textio, Jasper, Fireflies and Otter could assist in tasks such as meeting reports, lobbying and aid memoirs. He warned against trusting AI to generate zero drafts of resolutions or conventions due to potential algorithmic bias, but suggested that AI tools could provide alternative formulations to stuck diplomatic negotiations.

Ambassador Bhatla also emphasised the importance of interesting diplomatic reports, noting that AI may be able to help streamline executive summaries. Finally, he highlighted the importance of judgment and speaking truth to power in diplomacy, noting that these are essential aspects which AI cannot replace. He concluded that, “No AI, no, no, no data algorithms can replace the sort of like human touch, the human trust, and the fact that when you speak to someone, 80% of the signals is nonverbal.”:

At a panel session concerning the future of the diplomatic profession, Jovan Kurbalija discussed the implications of automation and AI on the field of diplomacy. Kurbalija opened his speech by pointing out the importance of diplomatic reporting, “starting from the ancient time from Akhenaten and his tables, Talamarna tables, you will find the first diplomatic archives, also archives.” 

Kurbalija discussed the potential uses of AI in diplomacy and identified certain tasks that AI cannot replace, “It’s true [that there are niches which cannot be automated], but I hope that during today’s discussion, we can dig and identify those niches.” He also noted that AI can be used to draft resolutions and generate constructive ambiguity, as well as identify cross-cutting linkages. Kurbalija also highlighted that “nuances and attribution of statements are extremely important” and that “AI cannot replace the nuances and attribution of human communication.”

Kurbalija concluded his speech on an optimistic note, “I'm more optimistic about the chances that we may get it right. I hope that we'll have a better reports. I hope that they will be more focused, and that we will crystallize our, when I say our human thinking, and reflections on issues around us, and ultimately contribute to the peace and more harmonious relations in the world.”  He proposed that a new skillset is required for the interplay between humans and AI, “The interplay between us and AI will be a new skill, new way of prompting, asking question, which is in a way Socratic dialogue, how to ask the question AI and what you will get from AI.”.

Irena Zubcevic addressed the importance and relevance of Voluntary National Reviews (VNRs) in relation to the implementation of the 2030 Agenda for Sustainable Development and the SDGs. Zubcevic emphasized that VNRs are voluntary and country-led, and should track progress, identify achievements, and gaps, mobilize means of implementation and partnerships, identify solutions, and be open and transparent for all people.

In regards to Artificial Intelligence (AI), Zubcevic stated that AI can be an excellent way to centralize and synthesize data, policies, and recommendations in order to minimize errors and analyze trends. AI can also replace people in doing straightforward research and reduce the reporting burden on national administration. However, AI cannot think out of the box or have creative thinking that can connect things to arrive to something better; a human brain is still necessary for this. 

Zubcevic expressed that user-friendly AI should be made more readily available and accessible, especially for developing countries with limited resources. She believes that “AI can be an excellent way to centralize data, policies, recommendations from different processes that can be available to anyone” and that “AI can replace people in doing straightforward research such as centralizing data, information, policies, recommendations, so people looking at the obstacles and gaps in implementation do not need to search for it.” 

In conclusion, Zubcevic believes AI can be of great assistance to countries and organizations, helping to reduce the reporting burden and streamline information. However, AI cannot completely replace humans, as creative thinking and judgment is still required for certain tasks. User-friendly AI should be made more accessible, especially for developing countries with limited resources.Katharina Hoene, the Director of Research at the Diplo Foundation, moderated a web debate on the use of artificial intelligence in diplomatic reporting. The panel of speakers included Dr. Johan Kubalier, the Director of Diplo, Ambassador Umesh Bhatia from Singapore, Irena Subcevich from Croatia, and Liz Galvest, a former British diplomat. 

The discussion focused on the role of artificial intelligence in diplomatic reporting and what can and cannot be automated. Hoene said: “We organise these debates once a month on the first Tuesday of the month and also in the context of the International Forum on Diplomatic Training. We do this to bring current issues and pressing topics regarding diplomatic practice into a conversation with practitioners, diplomats, people from outside and especially in this kind of debate we're bringing diplomats and technologists together.” 

Dr. Kubalier remarked that the question of AI replacing diplomats was quickly becoming a more concrete discussion, with a focus on what aspects could be automated to what extent and with what consequences. He suggested that rather than replacing diplomats, more of them may be needed to fact-check AI-generated reports. Subcevich offered insight from her experience as the focal point at the UN Department of Economic and Social Affairs for the voluntary national reviews on the implementation of the Sustainable Development Goals and said that AI tools could be used to help countries report on their progress towards the SDGs. Galvest, who had been in charge of reporting from the UN General Assembly for the UK mission, highlighted the importance of the human touch in diplomatic reporting. 

Hoene concluded by noting that automating diplomatic reporting would require the development of new skills and roles for diplomats, and that the “human touch and diplomatic gloss will still be key elements of diplomatic reporting.”:

In conclusion, the panelists engaged in a lively discussion concerning the use of Artificial Intelligence (AI) in diplomatic reporting. Participants suggested that while AI could be useful in certain tasks, such as collecting data, automating diplomatic reporting would require the development of new skills and roles for diplomats. Moreover, the human touch and diplomatic gloss will still be necessary to ensure accuracy and nuance in diplomatic reports. It was agreed that AI could be of great assistance to countries and organizations, helping to reduce the reporting burden and streamline information. However, AI cannot completely replace humans, as creative thinking and judgment is still required. It is essential that user-friendly AI is made more accessible, especially for developing countries with limited resources. Ultimately, the panel highlighted the importance of considering the implications of AI on diplomatic reporting and exploring how it can be best used in order to further diplomatic practices.