Data-Categorisation.txt

Data Categorization Document

1. Overview

This document provides detailed information on the data categorization process used in the Ruby app. It includes data structures, APIs, data flow, and implementation guidelines for modules that need to process or utilize data categories extracted from user messages.

2. Data Models and Structures

2.1. Data Entities

A. DataCategories

	•	Description: Contains categorized information extracted from a user’s message.
	•	Attributes:
	•	intent: String? (e.g., “seeking advice”, “expressing emotion”)
	•	emotions: [String]? (e.g., [“nervous”, “excited”])
	•	topics: [String]? (e.g., [“dating”, “confidence”])
	•	entities: Entities?
	•	preferences: [String]? (e.g., user preferences inferred from conversation)
	•	actionItems: [ActionItem]?
	•	timeReferences: [String]? (e.g., [“tomorrow”, “next week”])

B. Entities

	•	Description: Represents people, events, and places mentioned in the message.
	•	Attributes:
	•	people: [Person]?
	•	events: [Event]?
	•	places: [Place]?

C. Person

	•	Description: Represents an individual mentioned by the user.
	•	Attributes:
	•	name: String
	•	relationship: String (e.g., “friend”, “sister”)
	•	notes: String? (optional)

D. Event

	•	Description: Represents an event mentioned by the user.
	•	Attributes:
	•	name: String
	•	date: Date
	•	time: Date?
	•	location: String?
	•	notes: String?

E. Place

	•	Description: Represents a location mentioned by the user.
	•	Attributes:
	•	name: String
	•	address: String?
	•	notes: String?

F. ActionItem

	•	Description: Represents follow-up tasks or reminders resulting from the conversation.
	•	Attributes:
	•	description: String
	•	dueDate: Date?
	•	isCompleted: Bool

2.2. Updated Message Structure

struct Message {
    var sender: String // "user" or "Ruby"
    var timestamp: Date
    var content: String
    var messageId: String
    var dataCategories: DataCategories?
}

3. DataCategorizationModule

3.1. Purpose

Processes user messages to extract data categories for personalization and context.

3.2. Functions

	•	func analyzeMessage(message: String, completion: @escaping (DataCategories?) -> Void)

3.3. Dependencies

	•	May use GPT-4o API or other NLP services.

4. Data Flow

4.1. Message Processing Flow

	1.	User Inputs a Message:
	•	Captured by the ChatInterfaceModule.
	2.	Data Categorization:
	•	The message is sent to the DataCategorizationModule to extract DataCategories.
	3.	Storage:
	•	The Message along with its DataCategories is saved using the UserDataStorageModule.
	4.	AI Integration:
	•	The AIIntegrationModule uses the Message and its DataCategories to construct a prompt for GPT-4o.
	5.	Response Handling:
	•	The AI’s response is displayed to the user via the ChatInterfaceModule.
	•	Any new DataCategories extracted from the AI’s response (if necessary) are processed similarly.

5. Implementation Guidelines

5.1. Extracting Data Categories

Use the DataCategorizationModule to analyze messages.

Example Code:

func analyzeMessage(message: String, completion: @escaping (DataCategories?) -> Void) {
    // Construct the prompt for GPT-4o
    let prompt = """
    Analyze the following message and extract the user's intent, emotions, topics, entities (people, events, places), preferences, action items, and time references. Present the information in JSON format.

    Message: "\(message)"
    """

    // Call GPT-4o API
    aiIntegrationModule.sendPrompt(prompt: prompt) { response in
        // Parse the JSON response into DataCategories
        if let data = response.data(using: .utf8) {
            do {
                let dataCategories = try JSONDecoder().decode(DataCategories.self, from: data)
                completion(dataCategories)
            } catch {
                print("Error parsing data categories: \(error)")
                completion(nil)
            }
        } else {
            completion(nil)
        }
    }
}

5.2. Incorporating Data Categories into AI Prompts

When constructing prompts for GPT-4o, include relevant DataCategories to provide context.

Example:

var prompt = "You are Ruby, an AI companion..."

if let dataCategories = message.dataCategories {
    if let intent = dataCategories.intent {
        prompt += "\nIntent: \(intent)"
    }
    if let emotions = dataCategories.emotions {
        prompt += "\nEmotions: \(emotions.joined(separator: ", "))"
    }
    // Include other data categories as needed
}

prompt += "\nUser: \(message.content)"

6. Privacy and Compliance

6.1. User Consent

	•	Transparency: Inform users that their messages will be analyzed to extract data categories for personalized experiences.
	•	Opt-Out Option: Provide users with the ability to opt out of data categorization if they prefer.

6.2. Data Security

	•	Encryption: Securely store sensitive data, especially when it includes personal entities like people and events.
	•	Access Control: Limit access to data categorization information to necessary modules only.

7. Modules Utilizing Data Categorization

	•	AIIntegrationModule
	•	Incorporates DataCategories into prompts to GPT-4o for personalized responses.
	•	NotificationModule
	•	Schedules notifications based on actionItems and timeReferences extracted from messages.
	•	UserDataStorageModule
	•	Stores and retrieves DataCategories associated with messages.

8. Testing and Validation

	•	Unit Testing: Test the DataCategorizationModule with various sample messages to ensure accurate extraction.
	•	Integration Testing: Ensure that modules interacting with DataCategories handle the data correctly.

9. How to Use This Document

	•	For Developers:
	•	Refer to this document when working on modules that require data categorization.
	•	Use the data structures and implementation guidelines provided.
	•	For Updating Modules:
	•	If adding data categorization features to an existing module, update the module’s data models and APIs accordingly.

End of Data Categorization Document

Instructions for Use:

	•	When working on a module that requires data categorization, copy the relevant sections from the Data Categorization Document into your working environment or GPT session.
	•	Ensure consistency by adhering to the data structures and naming conventions specified.
	•	Update the shared documents if any changes are made to the data categorization process to keep all modules aligned.