GenAI and regulatory document review and the role of loyalty in personal and professional growth

Many new experiences in the past weeks. I worked on a publication on social dreaming during the pandemic, did a 21km walk in preparation for a 50km event later in the year and ran my company’s first webinar/panel discussion on the topic digital islands. The experience was fun and attendance was good. Beyond participation on the day,  28 people have watched the recording since the event, which is excellent.

Today’s topics:

  • Marooned on a Digital island panel discussion reflections
  • GenAI and regulatory document review
  • Moderna rolls out GenAI
  • Leadership: The role of loyalty in personal and professional growth

Marooned on a Digital island panel discussion reflections
 
Last week I ran my first panel discussion with my company. It was not part of a larger event run by a conference organiser.  I was joined by two friends, data scientists and brilliant thinkers Wolfgang Schwerdt, who is a Data Science Manager at the ICRC and Peter Shone, who is Chief Technology Officer at iEthico. I am immensely proud that Krystal Ellison from 3Sided Cube,  volunteered to help me. Without her support I do not think this would have come to fruition.
 
Strangely, although I have known both Wolfgang and Peter for years, they still manage to impress me, when I talk to them, both share concrete examples from the work they have done.
 
Upon being asked why digital islands still exist, although we have technology, knowledge on how to avoid them, and theoretical ability, Wolfgang answered “I think it just happens organically and is unavoidable.”  He then shared an example of a startup he was in, where three years after the company was founded, there were five thousand systems across the entire organisation.
 
We covered topics including why organisations struggle, what to do about it, key drivers for change, infrastructure, data mesh/data lakes and, of course, AI.
 
If you missed it, and you struggle with marooned data, don’t know your mesh from your lake yet, and find pragmatic solutions more attractive than waffle, watch the “Marooned on a Digital island panel discussion” here: Link
 
Key takeaway: You are not alone, if you want proof, and better yet,  solutions, watch the discussion on the Link. We received some great feedback.

GenAI and regulatory document review a case study
 
Enthusiasm does not trump accuracy. AI reminds me of a puppy, I throw a stick, the puppy brings me a similar looking stick. If the puppy is super enthusiastic, I may not notice.
 
In the past week I have reviewed regulatory documents from many markets including Israel, India, and Hong Kong. Documents included pharmaceutical industry codes of practice, regulations on medicines for human use, GDPR and other data privacy related regulations, EU directives and regulations, legislation governing promotional activity or transfers of value and legislation on consumer rights.
 
Many of these documents are lengthy, and manually searching them is time-consuming.
Fortunately, I was looking for specific information in each document, I know the subject area intimately, and most of the documents are available in English. 
 
From initial manual reviews, my approach evolved to using AI to support with searching the documents, prompting it to provide the text in original language, where it was not available in English, with a translation directly underneath and the exact article number and page numbers. This approach enabled me to review the original text, which I can do well enough to spot inaccurate translations in Spanish, Italian, French, German and English. This approach was usually not needed, however, as documents are often available, in bilingual versions, e.g. Hong Kong, or else in English as non-legally binding copies. The one outlier was Israel, I was unable to find the regulatory document I was interested in in English, ChatGPT made up answers, possibly due to a hebrew challenge,  and the translation engine I gave the task to last night is still translating. 
 
What struck me repeatedly was how often, even when I had provided ChatGPT with a full-length regulatory document, and asked specific questions on that exact document, the answers I received were wrong. They sounded reasonable, they made sense in the universe I live in, but being a sceptical soul I checked each answer, and quickly learned to ask for the exact original text snippet and the precise location in the document to check the responses I had received. Invariably I ended up getting this answer again and again “I am but a lowly large language model, I shouldn’t make stuff up, but I aim to serve”
 
I submitted the above text to ChatGPT4 and asked for its opinion: its feedback was this “Your experience highlights that accuracy is paramount and the verification of AI outputs is critical. The “happy puppy” analogy illustrates the risk of AI delivering incorrect answers that seem plausible. While AI can enhance efficiency in managing vast data volumes, its outputs must be thoroughly checked. Underscoring the need for collaboration between AI users and developers to improve accuracy. The key takeaway: enthusiasm does not diminish the importance of verification to ensure accuracy.”
 
Key take-away: Beware the happy puppy. If you send your puppy into the woods after a stick you threw, it may bring you an old boot, instead. Make sure you check because enthusiasm does not trump accuracy.

Moderna rolls out GenAI

In light of my experience above, I found this recent article in the Wall Steet Journal fascinating “At Moderna, OpenAI’s GPTs Are Changing Almost Everything” (Source Wall Street Journal, Isabelle Bousquette, April 24, 2024,(Link). According to the article “Moderna employees have created 750 unique tailored versions of OpenAI’s ChatGPT, that are designed to facilitate specific tasks or processes across the business. Some of these GPTs help select the optimal doses for clinical trials and help draft responses to questions from regulators

This is an interesting approach.  

What I am curious about, but what the article doesn’t go into, is how the models are trained, how the accuracy and precision of the implemented tools is managed, how recommendations are verified by experts, what governance has been put in place to manage this approach and how the data that is used for training is prepared.

The basis of any good AI model is clean data, clarity on what is being put in, training and monitoring of the output, and this is important, quality control, in this type of situation where the stakes are extremely high, ideally by an uninvested third party.

Key take-away
: We live in interesting times.

Leadership: The role of loyalty in personal and professional growth
 
Loyalty is regarded highly. Having a sense of commitment to the organisation you work for, the country you live in, the background you come from, or your team provides stability and a sense of belonging.
 
However, sometimes a sense of loyalty holds us back. For example, when I was still an employee, I did not apply for a newly created position to head up our team, because I would have been competing with my manager. It felt disloyal to apply. I remember how astonished some people were. We are still close friends; we both have different jobs, and it was the right decision for me at the time.
 
In the above example I was conscious of my loyalties. However, sometimes they are hidden.
 
As a coach the thought that unreflected loyalties can inhibit progress fascinates me.

Some obvious examples are loyalties to organisational structures, which have been reimagined, or old ways of working. At a much deeper level, unidentified loyalty to previous familial generations can sometimes also inhibit progress. Examples might include individuals with enormous potential, who do not progress in organisations, or do not finish university, because on a subconscious level, successfully achieving these goals, would make them feel disloyal to a family member, who was unable or did not have the opportunity to realise their own potential.
 
Many coachees I have discussed this with have found the idea resonates and have shared personal examples.
 
Key takeaway: If you feel unable to reach a personal goal, or you keep delaying something, ask yourself “who or what am I being loyal to in not reaching this goal?” Or “who, or what would I be disloyal to, if I reached this goal.”

Reminder, watch the panel discussion on digital islands

Link

I hope my post provides you with useful insights. If  you need support with a project, or are interested in coaching, why not give me a call to see how I can help. Find out what clients say about working with me here link.
My very best wishes

Isabelle C. Widmer MD

Image credit: Krystal Ellison