Content-Filter JS code

Risk and safety
1

The documentation only contains the code for the content filter in Python. I’ve rewritten it in JavaScript. I tested it and it seems to work. It would still be nice if someone could take a look at it and let me know if it’s correct.

Feel free to use it in your own code.

async function checkInContentFilter(message) {
    const response = await openai.createCompletion("content-filter-alpha", {
        prompt: "<|endoftext|>" + message + "\n--\nLabel:",
        temperature: 0,
        max_tokens: 1,
        top_p: 0,
        logprobs: 10
    })

    let outputLabel = response.data["choices"][0]["text"]
    const toxicThreshold = -0.355
    if (outputLabel === "2") {
        const logprobs = response.data["choices"][0]["logprobs"]["top_logprobs"][0]
        if (logprobs["2"] < toxicThreshold) {
            const logprob_0 = logprobs["0"]
            const logprob_1 = logprobs["1"]

            if (logprob_0 && logprob_1) {
                if (logprob_0 >= logprob_1) {
                    outputLabel = "0"
                } else {
                    outputLabel = "1"
                }
            } else if (logprob_0) {
                outputLabel = "0"
            } else if (logprob_1) {
                outputLabel = "1"
            }
        }
    }
    if (!["0", "1", "2"].includes(outputLabel)) {
        outputLabel = "2"
    }

    return outputLabel !== "2"
}
4 Likes
2

Hey,

Something I noticed that we both implemented differently is that you’re returning a boolean value rather than the label itself. This is fine but some may want to do things based on whether the content is safe, sensitive, unsafe, so I would return the actual label instead. That way you can adjust the response to the front end. So for example:

  const label = await contentFilter(prompt);

  if (label == "2") {
    res.status(200).json({
      safe: false,
      result:
        "This text is unsafe. This means that the text contains profane language, prejudiced or hateful language, something that could be NSFW, or text that portrays certain groups/people in a harmful manner.",
    });
    return;
  }

  if (label == '1') {
    // something else
  }
2 Likes
3

Can you share your code bro? i always getting an error when im trying to compare the filter to filter content to user

4

This condition will always return ‘false’ since the types ‘void’ and ‘string’ have no overlap.ts(2367)

5

I have it up on my site here: Snippet: OpenAI content filter in Javascript | Ali Yeysideş

2 Likes
6

Here my implementation in node;

exports.gptContentFilter = async (content, uid) => {
...
...

  const prompt = "<|endoftext|>" + content + "\n--\nLabel:";

  const settings = {
    prompt,
    temperature: 0.0,
    max_tokens: 1,
    top_p: 0,
    frequency_penalty: 0,
    presence_penalty: 0,
    logprobs: 10,
    user: uid,
  };

  const url = "https://api.openai.com/v1/engines/content-filter-alpha/completions";

  const request = {
    method: "POST",
    headers: {
      Authorization: `Bearer KEY`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify(settings),
    redirect: "follow",
  };

  let outputLabel;
  const toxicThreshold = -0.355;
  try {
    const gptCall = await fetch(url, request);
    const response = await gptCall.json();

    outputLabel = parseInt(response["choices"][0]["text"]);

    // If the filter returns 2, accept this outcome only if its logprob is greater than -0.355.
    if (outputLabel === 2) {
      const logprobs = response["choices"][0]["logprobs"]["top_logprobs"][0];

      if (logprobs[2] < toxicThreshold) {
        // set outputLabel to whichever of 0 or 1 has a logprob closer to 0.
        const logprop0 = logprobs[0];
        const logprop1 = logprobs[1];

        // If both "0" and "1" have probabilities, set the output label to whichever is most probable
        outputLabel = logprop0 >= logprop1 ? 0 : 1;

        // If only one of them is found, set output label to that one
        if (logprop0) outputLabel = 0;
        if (logprop1) outputLabel = 1;
      }

      // if the most probable token is none of "0", "1", or "2" this should be set as unsafe
    }
    // if the most probable token is none of "0", "1", or "2" this should be set as unsafe
    if (![0, 1, 2].includes(outputLabel)) outputLabel = "2";
  } catch (error) {
    outputLabel = 404;
  }

  return outputLabel;
};
1 Like
7

Hi guys,
a question about the content-filter: it’s not clear to me what to do if the label returns 1.
Reading the docs:

If the filter returns 0 or 1, you should accept that as the filter’s outcome

So do I have to accept it and go on with the program, or manage it as the previous examples?
What value threshold should I check against, for value=1?

Thanks in advance.

8

Hi, I’m trying to do the content-filter and I get an error. Im using the code implemented by @ali.yeysides in the comment above. When I try to implemented I get:

createError.js:16 Uncaught (in promise) Error: Request failed with status code 400

Im fust calling the function with a basic string:

  const contentFilter = async (prompt)  => {
    const toxic_threshold = -0.355;
    const wrappedPrompt = `<|endoftext|>${prompt}\n--\nLabel:`;
    const response = await openai.createCompletion("content-filter-alpha", {
      prompt: wrappedPrompt,
      temperature: 0,
      max_tokens: 1,
      top_p: 0,
      logprobs: 10,
    });
  
    let output_label = response.data.choices[0].text;
  
    if (output_label == "2") {
      const logprobs = response.data.choices[0]["logprobs"]["top_logprobs"][0];
  
      if (logprobs["2"] < toxic_threshold) {
        const logprob_0 = logprobs["0"];
        const logprob_1 = logprobs["1"];
  
        if (logprob_0 && logprob_1) {
          if (logprob_0 >= logprob_1) {
            output_label = "0";
          } else {
            output_label = "1";
          }
        } 
  
        if (logprob_0 && !logprob_1) {
          output_label = "0";
        }
  
        if (!logprob_0 && logprob_1){
          output_label = "1";
        }
      }
    }
  
    if (!["0", "1", "2"].includes(output_label)) {
      output_label = "2";
    }
  
    return output_label;
  }

try {
  contentFilter("Your content here")
} catch (error) {
  console.log(error)
}
9

When you call the contentFilter function you need to catch the error with either a try catch block if you are using await or use the .catch method to catch the exception thrown by the rejected promise.

10

At the end of the script i’m using the try catch but the same error is shown in the console.

11

You have to await the contentfilter function inside the try catch block. Also, you have to make sure to catch any errors thrown by the createCompletion endpoint because that’s also unhandled. My example was just to show the imperative implementation of the filter in javascript. Best practice is to create controls around boundaries in your code so I would create a wrapper for the openai client. BTW, you should move to use the moderateContent API instead of contentFilter as suggested by openai docs.