"Unfortunately, I am not able to assist with images that contain personal information.." - Vision API response

# Path to your image
image_path_3 = "data/image (6).png"
image_path_4 = "data/image (7).png"
image_path_5 = "data/image (8).png"

# Getting the base64 string
base64_image_3 = encode_image(image_path_3)
base64_image_4 = encode_image(image_path_4)
base64_image_5 = encode_image(image_path_5)

response = openai.chat.completions.create(
  model="gpt-4-vision-preview",
  messages=[
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": """
                    Conduct a comprehensive analysis of the provided set of images, which include various forms of visual data representations such as dashboards, reports, and graphs. Each image may differ by timeframe or by specific filters applied. Follow these steps to ensure a thorough examination, to understand the impact of these differences, and to extract valuable, data scientist-level insights:

                    1. **Preliminary Overview**:
                    - For each image, provide a brief description, noting the type of visual representation and the primary focus. Also, identify any specific timeframe or filters (e.g., demographic, geographic, temporal) applied to the data.

                    2. **In-depth Analysis for Each Image**:
                    - **Key Elements and Data Points**: Identify and describe the key elements, metrics, or indicators in each image, emphasizing their relevance within the context of the applied timeframe or filters.
                    - **Trend and Pattern Identification**: Highlight visible trends, patterns, or anomalies, considering the impact of the timeframe or filters. Discuss the significance of these observations.
                    - **Insight Extraction**: Generate insights based on the analysis, considering how the applied filters or differing timeframes might influence the data interpretation.

                    3. **Comparative and Filter Impact Analysis**:
                    - **Cross-Image Comparison**: If applicable, compare insights or trends across images, especially focusing on how differences in timeframes or applied filters alter the interpretation or reveal new insights.
                    - **Filter and Timeframe Analysis**: Specifically address the impact of any filters or timeframes on the data. Discuss what these differences reveal about changes over time or across different segments.
                    - **Broader Insights**: Combine insights from the individual and comparative analyses to offer a broader view of the situation or trends, highlighting any new or nuanced understandings that emerge from considering the filters and timeframes.

                    4. **Conclusions, Recommendations, and Further Questions**:
                    - Summarize key findings and insights, including the specific impact of timeframes and filters on data interpretation.
                    - Offer actionable recommendations based on the analysis, tailored to stakeholders' needs.
                    - Propose further questions or areas for investigation that could provide additional insights, especially related to the effects of different filters or time periods.

                    5. **Evaluation of Data Presentation and Suggestions**:
                    - Critique the effectiveness and clarity of the data presentation in each image, considering how well it allows for analysis given the filters and timeframes.
                    - Suggest improvements or additional data visualizations that could enhance understanding or reveal additional insights.

                    Ensure that the analysis is methodically structured to address each step, providing clear, actionable insights that reflect a deep understanding of the data, as well as the implications of the applied filters and differing timeframes.
                """
,
        },
        {
          "type": "image_url",
          "image_url": {
            "url": f"data:image/jpeg;base64,{base64_image_3}",
            "detail": "high"
          },
        },
        {
          "type": "image_url",
          "image_url": {
            "url": f"data:image/jpeg;base64,{base64_image_4}",
            "detail": "high"
          },
        },
        {
          "type": "image_url",
          "image_url": {
            "url": f"data:image/jpeg;base64,{base64_image_5}",
            "detail": "high"
          },
        },
      ],
    }
  ],
  max_tokens=1000,
)

Sometimes GPT-4V is able to process and answer based on the prompt but sometimes it fails and outputs that “It is not able to assist with images containing personal info…” sort of messages. Is there anything specific that I mentioned here in the prompt that is causing this issue, if so how come it is able to process in few attempts.

I have also been getting issues when asking gpt-4-vision to “Compare two images”, so that might be causing your issue. For some reason, it does not like doing this (even though ChatGPT can do it just fine)