OpenAI API Onboarding Guide

On the day of the hackathon we'll be sponsoring your AI credits so that you can interact with GPT-3 and GPT-4 directly through the API endpoints. To do this, you'll need your OpenAI API key, which is your ticket to making requests to the API. This will be available on the day. In the meantime, feel free to explore the free ChatGPT 3 interface.

Creating a chat completion is the standard way to talk to GPT. The other methods, e.g. standard completions with text DaVinci 003, are deprecated.

You’ll be using your OpenAI key to make requests. You can either use a standard HTTP request library or install the Python or NodeJS libraries. Since we’re sponsoring your AI credits, we ask that you be mindful of how big your prompts are and how often you run a completion.

If you aren’t using an OpenAI library and need the API reference for chat completions, check it out here.
 

The various message types are useful to understand in order to craft a good prompt. You can order these messages however you want, but generally the system message will go first.

• System messages are input from you, the developer. You might use these to tell GPT what its identity is (e.g. “You are a question answerer”) and any constraints you want it to obey (e.g. “You only respond in French”). GPT4 adheres to system messages more strongly than GPT3.5.
• User messages are input from the end-user. You might use these to put in a user’s financial data, knowledge base query or other input.
• Assistant messages are messages that GPT gives back as responses. You can also use assistant messages to show GPT how you want it to behave by giving it an example assistant message in response to a user message. e.g. (User: Hello!, Assistant: Bonjour!, User: <actual input>, Assistant: <completion>).

There are two AI models that you’ll be using, and they each have their own strengths. Both of them work the same way, by receiving an array of text messages. You must convert any documents etc. you might want to use into text.

• gpt-3.5-turbo is very fast and very cheap. It can output hundreds of tokens in a couple of seconds. It’s really good at summarizing and extracting information.
• gpt-4 is very smart, but slower and more expensive. It’s really good at creative tasks and pattern recognition.

The best way to think about crafting the right prompt is to imagine you’re giving an assignment to somebody in grade 9.

• Be very clear with your expectations and requirements
• Use unambiguous language
• Give as much detail as possible

Sometimes a small tweak to a prompt can make a big difference, so it takes some trial and error. You can read about all the various prompting techniques here, but generally, you should get good results if you just take the time to be very clear about how you want the model to behave. You can even ask ChatGPT to write prompts for you if you need help.

GPT is nondeterministic: running the same prompt a second time will give different results. That’s because the default temperature (randomness) is 0.7. You can experiment with a temperature from 0 (completely consistent) up to 2 (completely crazy).

GPT works by predicting the most likely next token for a given prompt. A token is like a word, but can be smaller - long words are multiple tokens. In general, 75 words is about 100 tokens.

There are limits on how many tokens you can prompt GPT with and how many it can complete, and your API responses will contain information on how many tokens were used for prompts and completions. You can read more about tokens here.