How to write your own agents

We recommend that you place the code for all your agents in the agents directory (though it is not mandatory). All your submissions should contain an Agent class. We have added dummy agent example in dummy_agent.py and a api usage example in [prompt_agent.py]. The agent class should contain the generate_responses

How to participate in GPU Track

Set "gpu": true in aicrowd.json. While the gpu flag is set to true, the api will not be usable.

How to participate in Prompt Engineering Track

Set "gpu": false in aicrowd.json. API usage will be enabled only when GPU is not used.

Submission details

Add your agent class name in user_config.py as UserAgent

What does the Agent recieve as input and what outputs are expected

You will be provided conversations with 7 turns each in batches of upto 50 conversations. For each batch of conversations, the first set of turns will be provided to your model. After the response is receieved the further turns of the same conversation will be provided. Each conversation will have exactly 7 turns. Your model needs to complete all 7 responses of 50 conversations within **1 hour**. The number of batches of conversation your model will process will vary based on the challenge round.

Each batch will be of the following format:

Input 1
[
    {"persona B": ... "dialogue": ... }, # conversation 1  Turn 1
    ...
    {"persona B": ... "dialogue": ... }  # conversation 50 Turn 1
]

Model should return 50 responses for Turn 1

...
Input 7
[
    {"persona B": ... "dialogue": ... }, # conversation 1  Turn 7
    ...
    {"persona B": ... "dialogue": ... }  # conversation 50 Turn 7
]

Output format

The generate_responses function should return a dictionary with the following data

   "use_api": True/False 
   "prompts": [ list of the prompts that go as "messages" to the api ]            
   "max_generated_tokens": [ list of ints for the max generation limit on each cal l]
   "final_responses: [ <list of strings with the final responses> ]                 

Format for messages for each prompt should be as per OpenAI's multi message prompting format

  [
    {"role_1": role_name, "content": prompt},
    ... 
    
    {"role_n": role_name, "content": prompt},
  ]

## Example
  [    
    {"role": "system", "content": "You're a helpful assistant"},
    {"role": "user", "content": "Say this is a test"},
  ]     

When passing the final responses, the use_api flag should be set to false.

How to use the API?

Since the code isn't connected to the internet, we provide a passthrough to use the API by using the generate_responses function.

In short, the output of generate_responses can set the use_api flag and pass a set of prompts. The users

These are the steps to use the API

1. Set the use_api flag to True
2. Create the prompts for all the utterances and pass it as a list in the prompts key.
3. Specify the max_generated_tokens for each of the prompts, these can be different for each prompt.
4. The evaluator will run these prompts and return the results in the api_responses input argument in generate_responses.

Which model is used for the API

We use gpt-3.5-turbo-1106 with each prompt from a user role.