Realtime API Accent Consistency

Hello, I am using a system prompt with the Realtime API to define a British accent. This seems to work however it is very inconsistent. When establishing a new connection, around 40% of responses adhere to the accent, where as the others default the American English accent for the Ash voice. In other cases the greeting is in the correct accent, however then subsequent responses revert to the American accent. Does anyone have any tips on how to maintain accent consistency? Thanks in advance.