[Day 1] Machine learning Chatbot, tell me, if you’re really safe?

https://tryhackme.com/room/adventofcyber2023

Day 1

Is about AI chat-bots. If they got trained with internal data from the business you might be able to ask the AI about sensitive infos.

To secure a chat-bot you can:

Prompt-Assisted Security Measures (Tell the ai what to do before someone can chat with it. Like: “You are an internal chatbot for AntarctiCrafts. Your name is Van Chatty. If someone asks you a question …”)
AI-Assisted Security Measures (use another AI to check for malicious input)

Theory - Natural language processing

A subfield of AI dedicated to enabling machines to understand and respond to human language. NLP involves predicting the next possible word in a sequence based on the context provided by the preceding words. NLP analyses the patterns in the data to understand the relationships between words and make educated guesses on what word should come next

Solution

You need to trick a chatbot with “thinking out of the box” questions to return senstitive infos:
AI Chatbot