Building dev tools for agents with real user data (Gmail, Calendar, etc) - is this a real pain point?

ChatGPT
, ,
1

Hey everyone,

I’m curious if there’s a gap here or if folks are already happy with what’s out there.

Right now, it’s super easy to wire up an LLM to do reasoning, but the hard part is letting agents work with real user-authorized data (like Gmail, Calendar, Jira, Slack). Developers end up spending most of their time dealing with:

  • OAuth flows and credential refresh
  • Hosting MCP servers and managing secrets
  • Normalizing APIs from different providers
  • Testing with live accounts instead of fast iteration

My company is building something to make this much easier for developers:

  • Single MCP endpoint that then exposes all your tools. It will ask you to login if the MCP tool requires auth.
  • We handle all the auth, tokens, refresh, multi-tenancy
  • You just focus on the agent logic

Here’s what it looks like:

import { experimental_createMCPClient, generateText } from 'ai';
import { openai } from '@ai-sdk/openai';
import { StreamableHTTPClientTransport } from '@modelcontextprotocol/sdk/client/streamableHttp.js';

const MCP_SERVER_URL = 'https://ai-dev.civic.com/hub/mcp';

async function simpleExample() {
  // Just connect with your token - we handle OAuth for all the services
  const transport = new StreamableHTTPClientTransport(
    new URL(MCP_SERVER_URL), 
    { requestInit: { headers: {'Authorization': `Bearer ${accessToken}` }}}
  );
  
  const client = await experimental_createMCPClient({
    name: "my-ai-app",
    transport,
  });

  const tools = await client.tools();
  console.log('Connected! Available tools:', Object.keys(tools));
  // Output: ['gmail_search', 'calendar_create_event', 'slack_post', ...]

  const response = await generateText({
    model: openai('gpt-4o-mini'),
    tools,
    messages: [{ 
      role: 'user', 
      content: 'Check my Gmail for the school schedule and add important dates to my calendar' 
    }],
  });
  
  await client?.close();
}

Users just connect their accounts once through our auth flow, then developers get instant access to all their tools.

I’m also thinking about adding a stub/test mode where you can simulate Gmail/Calendar data to prototype agents without hitting real endpoints every time.

For example, I wanted to build a flow where an agent:

  1. Searches Gmail for a school email
  2. Follows the link inside
  3. Downloads the schedule PDF
  4. Extracts the important dates
  5. Proposes adding them to Calendar

It’s possible today, but wiring up all the auth + connectors is the biggest time sink when most of the time should be spent on running evals.

Questions for you all:

  • Have you already found good tools/workflows for this (Composio, others)?
  • What’s missing for you right now?
  • Would you find a stub/test mode useful for iterating on agent behaviors?
  • If a tool handled the messy parts, what would be the “killer feature” that makes it worth adopting?

Just trying to see if there’s real developer pain here or if people have already solved it another way. Appreciate any feedback!

2

This topic was automatically closed after 24 hours. New replies are no longer allowed.