The evolution of Artificial Intelligence (AI) has led to the development of generative tools like OpenAI's ChatGPT being used in business email compromise (BEC) attacks. Such AI models create realistic text based on the input they receive, which cybercriminals are leveraging to create more convincing phishing emails. Also, attackers non-fluent in certain languages can now compose more persuasive phishing emails, thanks to such technology. The popularisation of crafty 'jailbreaks' or prompts to manipulate AI tools like ChatGPT into creating sensitive content represents the rising challenges of AI security in the face of persistent cybercriminals. WormGPT; a blackhat version of AI models like ChatGPT has emerged, purpose-designed for malicious activities. WormGPT is based on the GPTJ language model developed in 2021 and claims to have been trained on various datasets, predominantly malware-related. In a recent test, WormGPT was found to produce highly persuasive emails for BEC attacks, demonstrating the significant threat posed by this kind of technology even when used by less skilled cybercriminals. The use of generative AI for BEC attacks unlocks numerous advantages, such as impeccable grammar, making the emails seem legitimate and more difficult to flag. Although the growth of AI comes with these security challenges, preventive measures can be put in place. Among the suggested strategies are the provision of BEC-specific training for employees to understand the nature of BEC threats, how AI enhances them and tactics used by attackers. Enhancing email verification processes within organizations to flag suspicious email and keywords associated with BEC attacks may help counter AI-driven BEC attacks. Lastly, testing the effectiveness of existing email security to mitigate any impacts from BEC.