Testing application around API calls with Jest in Typescript

API
1

I’m trying to write some tests around the API calls and how my application processes the results in Typescript. I’m using Jest as my test framework, and want to mock the OpenAI calls to make the tests reproducible and faster.

My problem with this right now is that I cannot get Jest to mock the openai module. That is, this code doesn’t actually mock the module at all. It leaves the OpenAI class as its original.

import { OpenAI } from "openai";

jest.mock('openai', () => {
    return jest.fn().mockImplementation(() => {
        return {
            completions: {
                create: jest.fn().mockImplementation(async () => { return mockOpenAIresponses.create; })
            }
        }
    })
});

Based on everything I read, the OpenAI class should get replaced with my mocked up class, but it does not. It still calls the original constructor.

I’ve found exactly one article implying that this works with openai, and many articles on Jest mocking in general which imply this should just work. Does anyone have a working example in typescript?

2 Likes
2

Damn! I also have the same issue. Have you been able to sort it out?

3

I’m still stuck. I just punted for now and have it skip my tests that call the OpenAI API.

4

Just as a sanity check, you are calling the Jest.mock function prior to any openai invocations correct?

5

Correct. I have exactly one test defined calling the API. Here’s a complete test script written with Jest. Everything I’ve read implies to me it should override the open AI API call. Instead, it calls the actual API.

Any advice on how to fix this will be greatly appreciated.

import {describe, test, expect, jest, beforeAll} from '@jest/globals';

// load the openai key into environment
import * as dotenv from "dotenv";
dotenv.config({ path: `.env.local`, override: true });

// mock the OpenAI calls  -- currently the jest.mock() call is not overriding openai library class.

// regular successful OpenAI response
const openai200normal = {
    "id": "cmpl-8Y1uU3RVsY9kkGnQrSE7rmWcdHNvk",
    "object": "text_completion",
    "created": 1703121186,
    "model": "gpt-3.5-turbo-instruct",
    "choices": [
        {
            "text": "This is a sample mockup.",
            "index": 0,
            "logprobs": null,
            "finish_reason": "stop"
        }
    ],
    "usage": {
        "prompt_tokens": 142,
        "completion_tokens": 42,
        "total_tokens": 184
    }
};

let mockOpenAIresponses = {
    create: openai200normal,
};

import { OpenAI } from "openai";

let mockedOpenAI;

async function call_the_api() {
    const openai = new OpenAI();
    const response = await openai.completions.create({
        model: "gpt-3.5-turbo-instruct",
        prompt: "Say this is a test",
        max_tokens: 7,
        temperature: 0,
    });
    return response;
}

describe('api call', () => {
    beforeAll(() => {
        jest.mock('openai', () => {
            return jest.fn().mockImplementation(() => {
                return {
                    completions: {
                        create: jest.fn().mockImplementation(async () => { return mockOpenAIresponses.create; })
                    }
                }
            })
        });
        mockedOpenAI = jest.mocked(OpenAI, { shallow: false });
    })

    // beforeEach(() => {
    //     mockedOpenAI.mockClear();
    // });

    test('call API to generate response', async () => {
        expect(await call_the_api()).toEqual(openai200normal);
    }); // 10 second timeout *should* be enough. I hope.
})
6

I’ve also tried it with the jest.mock() outside the beforeAll at the top level of the file.

7

I imported OpenAI after calling jest.mock and it worked for me.

const app = require("../index");
const request = require("supertest");

jest.mock('openai', () => {
  return jest.fn().mockImplementation(() => {
    return {
      chat: {
        completions: {
          create: jest.fn().mockImplementation(async () => {
            return { choices: [{ message: { content: "Hello, I am an AI" } }] };
          })
        }
      }
    };
  });
});
const OpenAI = require('openai');

Maybe you can try that and let me know how it goes.

2 Likes
8

Thank you, @Olyray for that better wording with an example :smile:

1 Like
9

Thanks for the reply.

I can’t use require in a Typescript file (possibly due to my jest configuration for ts-jest?). Rewriting my above example test script to be common javascript this works. This is a little bit inconvenient, but I can deal with it as a workaround for this one set of tests.

Trying the examples from the Jest documentation for ESM at /docs/ecmascript-modules I cannot get it to provide me an OpenAI on which I can call new. It does seem to successfully mock the module which is a step forward.

jest.unstable_mockModule('openai', () => {
    return jest.fn().mockImplementation(() => {
        return {
            completions: {
                create: jest.fn().mockImplementation(async () => { return mockOpenAIresponses.create; })
            }
        }
    })
});

const {OpenAI} = await import('openai');

When it calls new OpenAI(), it errors with this:

TypeError: OpenAI is not a constructor

Any clues on getting over this hurdle? I can then keep my tests in TS.

10

Try removing the curly braces in the OpenAI import. Also remove it in the file you’re testing to see if it works.

12

Hello Olyray,
Can you please share your full solution? For me mock is still trying to call the actual API.

13

Okay. This is it:

const app = require("../index");
const request = require("supertest");

jest.mock('openai', () => {
  return jest.fn().mockImplementation(() => {
    return {
      chat: {
        completions: {
          create: jest.fn().mockImplementation(async () => {
            return { choices: [{ message: { content: "Hello, I am an AI" } }] };
          })
        }
      }
    };
  });
});
const OpenAI = require('openai');

jest.mock('twilio', () => {
  return jest.fn().mockImplementation(() => {
    return {
      messages: {
        create: jest.fn().mockResolvedValue({ 
          body: 'Hello, I am an AI',
          numSegments: '1',
          direction: 'outbound-api',
          from: 'whatsapp:+14155238886',
          to: 'whatsapp:+2348179361570',
          dateUpdated: '2024-01-17T16:13:19.000Z',
          price: null,
          errorMessage: null,
          uri: '/2010-04-01/Accounts/AC13a822a2d16ec2b0561d0724bba5d5b4/Messages/SM6a41984b90a3e6d0d41ccedf4db87007.json',
          accountSid: 'AC13a822a2d16ec2b0561d0724bba5d5b4',
          numMedia: '0',
          status: 'queued',
          messagingServiceSid: null,
          sid: 'SM6a41984b90a3e6d0d41ccedf4db87007',
          dateSent: null,
          dateCreated: '2024-01-17T16:13:19.000Z',
          errorCode: null,
          priceUnit: null,
          apiVersion: '2010-04-01',
          subresourceUris: {
            media: '/2010-04-01/Accounts/AC13a822a2d16ec2b0561d0724bba5d5b4/Messages/SM6a41984b90a3e6d0d41ccedf4db87007/Media.json'
          } 
        })
      }
    }
  })
})
const twilio = require('twilio');

jest.mock('../conversationManager', () => {
  return {
    initializeDatabase: jest.fn(),
    getConversationHistory: jest.fn().mockResolvedValue([]),
    updateConversationHistory: jest.fn()
  };
});


describe("GET /", () => {
  it("Check that WhatsGPT is online", async () => {
    const res = await request(app).get("/");
    expect(res.statusCode).toBe(200);
    expect(res.text).toBe('WhatsGPT is running');
  });
});

describe("POST /message", () => {
  it("Ensure that the OpenAI API responds with the required message", async () => {
    const response = await request(app)
      .post("/message")
      .send({From: "whatsapp:+2348179361570", Body: "Hello"})
    const openai = OpenAI();
    const openaiResponse = await openai.chat.completions.create({ 
      messages: [{
        role: "system", 
        content: "You're a knowledgeable friend that your acquintances turn to for help. Your response should be brief. Use a single sentence if possible."
    }],
      model: "gpt-3.5-turbo",
     });

     const twilioClient = twilio();
     twilioResponse = await twilioClient.messages.create({
      body: "Hello, I am an AI",
      from: "whatsapp:+14155238886",
      to: "whatsapp:+2348179361570"
     });

     expect(openaiResponse.choices[0].message.content).toBe("Hello, I am an AI");
     expect(twilioResponse.sid).toBe('SM6a41984b90a3e6d0d41ccedf4db87007');
    //expect(response.text).toContain("Hello, I am an AI");
  })
});

I tried sharing the link to the repository, but apparently I’m not allowed to.

14

Thanks for posting this. In my case, openAI chat completions call is from within a function in another ts file. I think that is why it’s not getting mocked. In your case your test is making the call directly as opposed to my case where I have a function foo() which in turn calls chat.completions.create()

15

Then I guess it depends on how you mocked the function. You can console.log the OpenAI variable in the tested file to see if it is mocked. The original properties should be replaced with mocked ones.

16

For anyone that is using Python, you should check this out:

Full disclosure, I’m the author of that package.

17

This was the way that I mocked it and it worked pretty well:

// Mock the 'openai' module
jest.mock('openai', () => {
  return {
    // Mock the 'OpenAI' constructor
    default: jest.fn().mockImplementation(() => ({
      // Mock the 'chat.completions.create' method
      chat: {
        completions: {
          create: jest.fn().mockResolvedValue({ choices: [{ message: { content: 'response' } }] }),
        },
      },
    })),
  };
});
18

How do you use that within your test? It is not significantly different than from my original question. My problem is how to “activate” the mock inside my tests.

19

Here’s my full test file, seems like you’re missing the default property in the mock object which is the way how the new instance is mocked:

// Mock the 'openai' module
jest.mock('openai', () => {
  return {
    // Mock the 'OpenAI' constructor
    default: jest.fn().mockImplementation(() => ({
      // Mock the 'chat.completions.create' method
      chat: {
        completions: {
          create: jest.fn().mockResolvedValue({ choices: [{ message: { content: 'response' } }] }),
        },
      },
    })),
  };
});

describe('GptService', () => {
  let service: GptService;

  beforeEach(async () => {
    const module: TestingModule = await createCustomTestingModule({
      imports: [AiModule, CloudModule],
      providers: [GptService],
    }).compile();

    service = module.get<GptService>(GptService);
  });

  it('should be defined', () => {
    expect(service).toBeDefined();
  });

  describe('prompt', () => {
    beforeEach(() => {
      jest.resetModules();
      jest.restoreAllMocks();
    });

    it('should return a string', async () => {
      const result = await service.prompt('prompt');
      expect(result).toEqual('response');
    });
  });
});
1 Like
20

I had exactly the same problem as OP (TypeScript with ES6). After fiddling around various things, this (solutions @gallegodaniel9004 provided) has proven to work for me, with a few tweaks (perhaps we differ in some configurations).
So sharing my rather full setup in case others have the similar issues.

My main.ts

import OpenAI from 'openai' // without {} as suggested by official doc

// simplified implementation
export const main() {
    //...
    const openai = new OpenAI(...);
    const res = await openai.chat.completions.create(...);
    //...
};

to test in main.test.ts

import { jest } from 'jest';
import { main } from './main';
import OpenAI from 'openai';

// Mocking actual 'OpenAI' module
jest.mock('openai', () => {
  return {
    // This one is really important, otherwise you'll have "*.default is not a constructor error":
    // See: https://stackoverflow.com/a/61396377/2452628
    __esModule: true,
    // Mock the 'OpenAI' constructor
    default: jest.fn().mockImplementation(() => ({
      // Mock the 'chat.completions.create' method
      chat: {
        completions: {
          // need to specify type here to avoid "can't type assign to never" error
          create: jest.fn<() => Promise<any>>().mockResolvedValue(...),
        },
      },
    })),
  };
});

My jest.config.ts

const config: JestConfigWithTsJest = {
  // A preset that is used as a base for Jest's configuration
  // npm install -D 'ts-jest'
  preset: 'ts-jest',
  roots: [
    '<rootDir>/__test__',
  ],
  verbose: true,
};

My tsconfig.json

{
  "compilerOptions": {
    /* Visit https://aka.ms/tsconfig to read more about this file */
    "target": "ES2022",
    "lib": ["ES2022"],
    "module": "ES2022",
    "moduleResolution": "Node",
    "types": ["jest","node"],
    "sourceMap": true,
    "outDir": "dist",
    "preserveConstEnums": true,
    "allowSyntheticDefaultImports": true,
    "esModuleInterop": true,
    "forceConsistentCasingInFileNames": true,
    "strict": true,
    "skipLibCheck": true
  },
  "include": ["src/**/*", "__test__/**/*", "jest.config.ts"],
  "exclude": ["node_modules"],
}

Hope this helps

1 Like