I have been trying to get the openai python package in to my Lambda function. I keep running in to the error
Runtime.ImportModuleError: Unable to import module 'serverless_aws_lambda_sdk.internal_extension.wrapper': No module named 'pydantic_core._pydantic_core'
I have tried using Docker to make sure I am pip-ing the correct architecture both x86 and arm64 (while also changing my lambda arch). I am using the Serverless framework, along with a layer. From what I have found elsewhere it seems that using a Docker image should fix it, so maybe I am running into issues and not setting up my Docker correctly. I have tried using the pre-built layers found on GitHub (I can’t post a link), and this hasn’t worked either.
Has anyone found a good solution to trying to get openai package into Lambda?
I then zipped it and oploaded it to an AWS layer. The pip install parameters are critical to get the correct wheel files that are compatible with Lambda.
I have avoided installing the OpenAI Python SDK as a Lambda Layer, only because everything I do API-wise can be called by installing the requests layer. The cosine similarity stuff can be done on my own too without the SDK. I think the only thing you might need the SDK for is streaming.