Ingestion

Overview

The ingestion layer handles all data collection and context building for the Personalization API.
It transforms unstructured inputs (like emails, messages, or notes) into structured, queryable memory.

Once ingested, each record becomes part of the user’s behavioral embedding, accessible through summary and query endpoints.

Key Concepts

Event

An atomic user action or observed behavior.
Examples include:

• An email sent or received

• A message in Slack or Discord

• A calendar entry or task update

• A click, purchase, or decision event

Each event carries metadata (timestamps, type, participants, and content).

Document

A persistent artifact representing a snapshot of the user’s thinking or work.
Examples:

• Meeting notes

• Strategy documents

• To-do lists

• Written reflections or instructions

Documents are used to extract tone, phrasing, reasoning patterns, and preferences.

Signal

A lightweight or inferred observation derived from ongoing usage.
Examples:

• Time of day activity peaks

• Interaction frequency with certain people or tools

• User corrections or reactions

Signals may not be explicitly provided — they can also be inferred from other events.

Endpoint Reference

POST /ingest

Used to send events and documents into the system.

curl -X POST "https://api.fastino.ai/ingest" \
  -H "Authorization: x-api-key: sk_test_123" \
  -H "Content-Type: application/json" \
  -d '{
    "user_id": "usr_42af7c",
    "source": "gmail",
    "events": [
      {
        "event_id": "evt_1",
        "type": "email",
        "timestamp": "2025-10-10T09:00:00Z",
        "metadata": {
          "from": "ceo@company.com",
          "to": ["ash@example.com"],
          "subject": "Shift weekly to afternoons"
        },
        "content": "Let’s move stand-up to 2 PM starting next week."
      }
    ],
    "documents": [
      {
        "doc_id": "doc_7",
        "kind": "note",
        "title": "Work style",
        "created_at": "2025-09-29T18:00:00Z",
        "content": "Typical focus blocks 9–12 PT; meetings after 1 PM preferred."
      }
    ],
    "options": { "dedupe": true }
  }'

Response

{
  "ingested": { "events": 1, "documents": 1 },
  "skipped": [],
  "updated_at": "2025-10-11T16:05:05Z"
}

Schema Reference

Event Object

• event_id: unique string ID

• type: event category (email, message, task, etc.)

• timestamp: ISO 8601 UTC format

• metadata: optional structured info (e.g. sender, subject, tags)

• content: text body or summary

Document Object

• doc_id: unique string ID

• kind: document type (note, summary, log, etc.)

• title: optional name or heading

• created_at: creation timestamp

• content: main text body

Deduplication Logic

When options.dedupe is set to true, Fastino automatically ignores any record that:

• Has a matching event_id or doc_id

• Is identical in content and timestamp to a previously stored record

This ensures you can re-run ingestions safely without duplicating user context.

Updating Existing Data

Ingestion is idempotent.
Re-ingesting a record with the same ID updates its content rather than creating duplicates.

{
  "options": { "dedupe": true, "update_existing": true }
}

Use this when syncing from third-party APIs (e.g., Gmail or Notion) to keep user context continuously fresh.

Example: Ingesting Multi-Source Data

{
  "user_id": "usr_42af7c",
  "source": "workspace",
  "events": [
    {
      "event_id": "evt_meeting_12",
      "type": "calendar_event",
      "timestamp": "2025-10-10T16:00:00Z",
      "content": "Weekly product sync with the design team."
    },
    {
      "event_id": "evt_msg_45",
      "type": "slack_message",
      "timestamp": "2025-10-10T16:10:00Z",
      "content": "Let’s finalize the prototype before next Tuesday."
    }
  ]
}

This ingestion builds a timeline that informs routine prediction, communication style, and context handoff.

Relationship to the User World Model

Each ingestion expands the user’s world model with new behavioral and contextual data:

Performance & Limits

Tip: For higher volumes, batch ingestions and enable background deduplication via the dashboard.

Example Integration (Python)

import requests

BASE_URL = "https://api.fastino.ai"
headers = {
  "Authorization": "x-api-key: sk_test_123",
  "Content-Type": "application/json"
}

payload = {
  "user_id": "usr_42af7c",
  "source": "workspace",
  "events": [
    {
      "event_id": "evt_12",
      "type": "calendar_event",
      "timestamp": "2025-10-10T16:00:00Z",
      "content": "Weekly sync meeting with the design team."
    }
  ]
}

r = requests.post(f"{BASE_URL}/ingest", json=payload, headers=headers)
print(r.json())

Best Practices

• Keep events atomic and meaningful (avoid batching multiple unrelated actions into one event).

• Include timestamps and types for every record.

• Use source identifiers consistently (gmail, notion, slack, custom).

• Enable dedupe to prevent redundant data.

• Periodically clean old test data with /users/delete in sandbox mode.

Summary

The ingestion system is the input layer of the Fastino Personalization API.
It transforms scattered user activity into structured, retrievable signals that feed the world model.
A well-designed ingestion pipeline enables more accurate personalization, better predictions, and richer agent reasoning.

Next, continue to Privacy & Data Lifecycles to learn how Fastino manages user data retention and deletion.