Rules to Better GPT - 15 Rules

ChatGPT is a user friendly way to interact with OpenAI's GPT models. Essentially, it takes a model of GPT that is fine tuned for dialogue and turns it into a chat bot that responds in a human-like manner.

ChatGPT is a conversational language model from OpenAI, accessible both on web and mobile that makes it easy to have conversations with OpenAI's GPT models, and adds lots of cool features on top, allowing your conversations to be much more natural.

It also saves your message history so that you can go back and look at your conversations later!

Why is ChatGPT cool?

Figure: ChatGPT's response when asked "Why is ChatGPT cool?"

ChatGPT is an incredibly powerful tool. GPT-4 is a noticeable improvement over GPT-3.5, and we can only expect the subsequent models to be even more impressive. For now, it does have its skills and weaknesses.

Here are the most important things for which we know it to be useful (and not):

✅ Use it for

1. Writing

It can draft emails, reports, proposals, writing projects, and templates, as well as proofread and edit documents.

2. Summarization

It can create concise summaries of meetings or lengthy documents/text.

3. Coding

It can assist with coding tasks by offering suggestions, debugging code, generating code snippets, and providing explanations for programming concepts.

4. Creativity

It can generate new ideas, offer creative suggestions, and provide guidance on resolving conflicts or addressing challenging situations.

5. Content

It can generate engaging content for various platforms, including social media, presentations, etc.

6. Critiquing

It can provide constructive feedback on various forms of work (in text format only), such as writing, presentations, and code.

7. Learning

It can provide resources, explanations, and tutorials on almost any topic.

8. Research

It can gather information on various topics, analyze data, conduct SWOT and competitive analyses, and weigh pros and cons of different options.

9. Translation

It can help in translating text between languages, providing quick and reasonably accurate translations for both personal and professional use.

And there are also areas where it performs poorly.

❌ Dont use it for

1. Non-textual information

It is not designed to process or generate images, audio, or other non-textual information.

2. Mathematics

While it can handle basic arithmetic and some mathematical concepts, it is not designed for advanced or complex mathematical problem-solving.

3. Emotion and empathy

It can mimic emotions, but it doesn't truly experience or understand them, which can lead to inappropriate or tone-deaf responses.

4. Context understanding

It may struggle with understanding context and maintaining continuity when a conversation gets long. 🤖

GPT is a large language model developed by OpenAI with major financial backing from Microsoft, and is currently leading the market in terms of technology and capability.

AI technologies have been being developed at companies like Amazon, Google and Microsoft for some time. The first large language model, GPT-1, was released by OpenAI in 2018.
It wasn't until the release of ChatGPT, which is powered by GPT, that the technology became widely known.

Isn't it called ChatGPT?

Whilst sharing similar names ChatGPT and GPT are different.
GPT stands for Generative Pretrained Transformer and refers to the large language model that underlies the ChatGPT frontend. This model accessed using an API.

This allows for developers to build their own applications leveraging the same capabilities of ChatGPT over the top of the GPT model.A great example of this is the SSW RulesGPT bot.

Why is GPT cool?

Figure: ChatGPT's response when asked "Why is GPT cool?"

How does GPT work?

GPT models are trained on a huge amount of data (often refered to as parameters) that it can then use to predict what words should come next when given an input.
For example, if the training data contains a weather report talking about beautiful clear blue skies, when you ask it what color the sky is it can call back to that data and respond blue.

GPT is trained on a huge amount of parameters.

• GPT-4: 1.76 trillion parameters.
• GPT-3: 175 billion parameters.
• GPT-2: 1.5 billion parameters.
• GPT-1: 117 million parameters.

Strengths

• The model can respond to queries using natural language.
• The model can handle grammatical and spelling errors.
• Responses can come from the model very quickly - no need to access a database to retrieve data.

Limitations

• The model is only as good as it's training data and has no knowledge of anything outside of this data.

Try it out in the playground

GPT and ChatGPT are similar tools that serve slightly different purposes. Each of them has strengths and weaknesses so it's crucial to understand the right time to use them.

ChatGPT is strong at helping an individual do a specific task. For example, a user could could get responses to simple tasks like:

• "Write me a blog post on EF Core Code-First Migrations"
• "How do I create a table with EF Core?"
• "Can you help me find information about Northwind?"
• and much more...

On the other hand, GPT is better for tasks that require automation or programmatic integration. For example, it could be used to create systems for:

• Summarizing a page of text
• Tailoring inputs to output in a specific manner e.g. input a url, and it outputs an SSW rule
• Changing email content to fit a specific audience e.g. Product Owner, Developer, Designer etc
• and much more...

Additionally, GPT provides an API that developers can leverage.

Video: ChatGPT vs. GPT - Which Is Better? - Q&A Comparison (2 min)

GPT is an awesome product that can do a lot out-of-the-box.
However, sometimes that out-of-the-box model doesn't do what you need it to do.

In that case, you need to provide the model with more training data, which can be done in one of two ways.

Usually, for common scenarios GPT will already be adequate, but for more complex or highly specific use cases it will not have the required training to output what you need.

System Prompt

The system prompt is a prompt that is sent along with every request to the API, and is used to tell the model how it should behave.Using the system prompt is the easiest way to provide additional data to GPT, but there are also some downsides to this approach.

Benefits

• Easy to implement
• No extra setup cost
• Data can be easily changed or removed

Disadvantages

• The system prompt counts towards total token count - not suitable for large amounts of data
• Large system prompts can limit the amount of tokens available for questions and responses
• Adds extra cost to each request
• Potential for inconsistency depending on what data is sent

Fine Tuning

OpenAI provides a way for you to train new data into the model so that it is always available, without having to provide it with each request.

For example, if you want to build an app that outputs SSW rules based on a title, the untrained model probably won't know what SSW Rules are so you need to train it.

Benefits

• Suitable for larger amounts of data
• No extra cost per request as trained data lives on the server
• Consistent as trained data is available to all requests

Disadvantages

• Harder to implement
• Extra setup cost to fine tune the model
• Model needs to be fine tuned again to change or remove data
• Fine tuning may not be available for every model

ChatGPT is an extremely useful tool for software developers as it has been trained to understand how code functions. It will provide an invaluable alternative to StackOverflow, being a great resource to help developers troubleshoot.

ChatGPT can be used for:

• Detecting bugs in your code
• Solving compile time or runtime errors
• Generating code based on a text description
• Explaining how a piece of code works
• Translating code to a different language (e.g. Python code to C#)
• Minimising generation of boilerplate (e.g. JSON to C# classes)
• Helping automate the code refactoring process
• Helping perform code reviews

Try it yourself, copy and paste this into ChatGPT

What does this code do?
[HttpPut("{id}")]
public async Task<IActionResult> MoveRight(string id)
{
	try
	{
		if (await _legalApiDbContext.ParaLefts.Where(a => a.ParaId == id).CountAsync() != 0)
		{
			ParaLeft toDelete = _legalApiDbContext.ParaLefts.Where(para => para.ParaId == id).First();
			_legalApiDbContext.ParaRights.Add(new ParaRight { ParaId = id });
			_legalApiDbContext.ParaLefts.Remove(toDelete);
			await _legalApiDbContext.SaveChangesAsync();
			return Ok();
		} else
		{
			return StatusCode(StatusCodes.Status404NotFound);
		}
	}
	catch (SqlException err)
	{
		_logger.LogError(err.Message);
		return StatusCode(StatusCodes.Status500InternalServerError);
	}
}

Sometimes, a developer runs into a complex error and they start googling all over the internet to fix the issue. In strange or unlucky cases this can result in hours of searching.

ChatGPT helps smooth this process. Simply paste the error into ChatGPT and it will give you instant feedback about the problem.

For example, let's say you try to run ef migrations using the command:

dotnet ef database-update

If this command gives you an error like:

No project was found. Change the current working directory or use the --project option.

Then you could ask ChatGPT and it would give you the solution!

Try it yourself, copy and paste this into ChatGPT

What is this error in EF Core?
No project was found. Change the current working directory or use the --project option.

Integrating ChatGPT with your personal assistant, such as Siri for iPhone or Google Assistant for Android, can greatly enhance your mobile experience. This article will guide you through the steps to connect ChatGPT with Siri and Google Assistant.

Connecting ChatGPT with Siri (for iPhone)

Video: Siri is AWESOME when combined with ChatGPT! (I'll show you how!) (12 min)

Connecting ChatGPT with Google Assistant (for Android)

At the time of writing, Android integration is a little trickier, but still totally doable!Follow these instructions and you should be talking to your new AI assistant in no time.

Adding ChatGPT as an app on your Windows, Android, or iOS device can be extremely useful for a number of reasons.

For example, you can use ChatGPT to ask questions, seek advice, or just have a casual chat whenever you need it. Whether you're looking for information on a specific topic, need help with a task, or simply want to pass the time, ChatGPT is always available to assist you with its intelligent responses.

Furthermore, ChatGPT is constantly learning and improving its abilities, which means that the more you use it, the more personalized and accurate its responses will become. With ChatGPT as an app on your device, you can have an intelligent and reliable virtual assistant at your fingertips, ready to help you with whatever you need, whenever you need it. So, whether you're a busy professional or just someone looking for a fun and engaging chatbot, adding ChatGPT as an app is a decision that you won't regret.

How to install ChatGPT on Mobile

• Android - https://play.google.com/store/apps/details?id=com.openai.chatgpt
• iOS - https://apps.apple.com/us/app/chatgpt/id6448311069

How to add ChatGPT as an Edge app:

AI can be helpful, and in some situations, it is good to be transparent that it helped you. Other times, however, it is more important that the message feels like it came personally from you.

In the case that you want to be transparent, you should indicate when AI was used.

When you want to be transparent (generally internal communication), there are benefits:

• Adoption: Spread and encourage usage of AI by showing how and when you use it, enabling others to think about when they can use it too
• Transparency: Others can identify AI-generated content, promoting openness and trust
• Responsibility: Encourages responsible use of AI-generated content

For generated text, a good option is to use the 🤖 emoji.You should use it at the end of your message/email/etc. There are 2 options:

1. Use 🤖 when you used ChatGPT but you edited the result

   Example: “Improve SEO with these tips: Create quality content, optimize for mobile, and use relevant keywords. 🤖”

2. Use 🤖⭐ when the content is entirely generated by ChatGPT

   Example: "As we watched the sunset from our office balcony, the conversation turned to the latest tech trends. The sky was painted in hues of pink and orange, but our minds were on the latest software updates and hardware releases. The birds chirped their goodnight songs as we debated the merits of different programming languages and platforms. It was a peaceful moment, a brief pause in the chaos of the tech world, and we couldn't help but feel grateful for the opportunity to work in such an exciting and constantly evolving industry.🤖⭐”

This is also important for generated images, Meta has developed invisible watermarks that can be used to indicate that an image was generated by AI.
https://aibusiness.com/responsible-ai/meta-develops-invisible-watermarks-to-track-ai-image-origins

Google has also done something similar. Check out SynthID

ChatGPT has an awesome API and Azure services that you can easily wire into any app.

The ChatGPT API is a versatile tool capable of far more than just facilitating chat-based conversations. By integrating it into your own applications, it can provide diverse functionalities in various domains. Here are some creative examples of how you might put it to use:

1. Automated Content Creation: Whether you’re generating blog posts, creating ad copy, or even writing a novel, the API can help streamline your creative process.
2. Document Editing: The API can be integrated into word processors or documentation software to provide real-time suggestions, corrections, or even automatic content creation.
3. E-Learning Platforms: From language learning apps to science tutoring software, you can utilize the API to create interactive, personalized learning experiences.
4. Idea Generation Tools: Build a tool that interacts with the API to brainstorm innovative ideas, from business strategies to home decoration concepts.
5. Coding Assistants: Whether it’s auto-generating pseudocode, suggesting code snippets, or providing guidance on best practices, you can create a valuable tool for programmers.
6. Smart Home Automation: Enhance your smart home application by integrating ChatGPT to handle complex routines, provide usage suggestions, or interact with users more naturally.
7. Project Management Software: Implement a smart assistant in your software that can help users plan, manage risks, or even generate project reports.
8. Healthcare Apps: The API can be used to understand medical terminologies, describe symptoms, or provide basic health and wellness advice.
9. Financial Management Tools: Integrate the API into your finance app for budget planning assistance, explaining complex financial terms, or even generating financial forecasts.
10. Mental Health Apps: Use the API to guide mindfulness exercises, provide motivational messages, or suggest stress relief activities.
11. Travel Planning Applications: Have the API generate itineraries, suggest interesting places, or provide information on local customs and etiquette. These examples only begin to explore the potential of the ChatGPT API. The key is to think creatively about how this powerful language model can be leveraged to meet your specific application needs.

A chatbot is a computer program that uses artificial intelligence to engage in text or voice conversations with users, often to answer questions, provide assistance, or automate tasks. In the age of Generative AI, good chatbots have become a necessary part of the User Experience.

Choosing the right chatbot service for your website can be a challenging task. With so many options available it's essential to find the one that best fits your needs and provides a good experience for your users. But what distinguishes a good chatbot from a great one? Here are some factors to consider.

Factors to consider

Custom Built vs 3rd Party Service

• Control vs Effort: Custom built provides more control but incurs high development effort & cost
• Cost per message: may be lower in custom built solutions for high volume of traffic

Performance

• Responses: Smooth and natural responses that answer questions while understanding context.
• Visual Design: Aligns with brand aesthetics.
• Content Tailoring: Adapts responses to fit brand voice.

Development Effort

• Pre-built Templates: Speeds up deployment.
• Drag-and-Drop Builders: Simplifies creation without coding.
• Documentation & Support: Assists in development.

Ease of Access

• Multi-Platform: Seamless functionality on desktop and mobile.
• Messaging App Integration: Works with apps like WhatsApp, Messenger.
• Language Support: Multilingual for global reach.

Research and Training

• API Support: API integration for third-party applications.
• Data Syncing: Real-time synchronization for up-to-date responses.

Scalability

• Traffic Management: Handles varying user traffic levels.
• Data Storage: Manages increasing user data.

Handling Curveballs

• Adaptive Responses: Adjusts to unexpected user inputs.
• Feedback Loop: Improves from past interactions.
• Human Agent Referral: Transfers smoothly to a human if needed.

Resilience to Trick Questions

• Response Filtering: Identifies misleading questions.
• Learning from Mistakes: Improves trick question handling.

Comparing platforms

The first decision is to choose between using a 3rd party chatbot service such as ChatBase/Botpress vs developing your own from scratch using the OpenAI API. The table below gives a summary of how these two options compare.

Factor	OpenAI API	3rd Party
Development effort and cost	Very High	Low
Control	Very High	Moderate
Cost to Train	Very Low	Very Low
Knowledge Base Limits	Unlimited	Limited but Sufficient
Cost per Message	Moderate	High

Before delving deeper into the comparison it would help to first understand the steps involved in building chatbots using either technology.

Steps to create chatbot using a 3rd Party service

For e.g. in Chatbase you would:

1. Train the bot by providing links to your web pages or by uploading docs
2. Craft a pointed prompt to give it instructions on how to respond to the user. For e.g. you can ask it to share URLs to your web pages when appropriate
3. Choose a GPT model – GPT-4 highly recommended
4. Embed an iframe or javascript code provided by the service on your website
5. Configure the chatbot for features such as a greeting msg, company logo, chat bubble colours, etc.

Steps to create chatbot using the OpenAI API

The following provides a very high level description of creating a chatbot from scratch using the OpenAI API. For a more in-depth explanation please watch:

Video: Exploring the Capabilities of ChatGPT | Calum Simpson | User Group (132 mins)

1. Convert your knowledge base into embeddings
2. Store embeddings and their corresponding text content in a vector database

3. Setup a server that can do the following

   1. Convert user query into an embedding
   2. Lookup vector database to find embeddings that are closest to the embedding created out of user query
   3. Insert the content corresponding to the matching embeddings into the OpenAI System message
   4. Pass recent user chat history to the OpenAI API
   5. Wait for OpenAI to generate a response. Present the response to the user.
4. Create a chatbot front-end widget

As you can see, developing a chatbot from scratch using the OpenAI API requires significant development effort and expertise. 3rd party chatbots on the other hand are much easier to program, embed on your website, and integrate with the Generative AI backend. As a rough estimate assume it will take a developer 20 days to build a custom chatbot from the ground up. If the developer costs $1000/day, this totals to $20K up front cost. Assuming a $399/month subscription of Chatbase on the other hand, it would take the custom solution over 4 years just to break even.

However, custom built chatbots provide a lot more control in how you train the AI model, what content you match the user query with, and what system message you provide the GPT engine to respond to a user’s query. You don’t get this level of control with 3rd party chatbots. The backend of custom built solutions can also be leveraged to serve multiple chatbots supporting completely different use cases. For e.g. one chatbot could provide basic company info to visitor’s on the company website, while a second chatbot could help employees find info on the company Intranet.

Cost to train the chatbot on your knowledge base is very inexpensive in both options. For example, you can train a chatbot on ~3000 pages for less than $1 USD using the OpenAI Embeddings model. Another factor is any limits on the size of the knowledge base. While there’s a limit on this in 3rd party chatbots the size should normally be more than enough. For e.g., Chatbase provides you 11M characters, which roughly equates to ~3500 pages of text of 600 words each.

The cost per message is generally much higher in 3rd party services. For e.g., as of Sep 2023, the highest paid plan in Chatbase that uses GPT-4 only gives you 2000 msgs/month for $399. If you exceed this limit, it gives you the option to use your OpenAI API key, but doesn’t disclose if this will cost exactly the same as what OpenAI charges. So for very high volumes of chatbot traffic, it’s worthwhile to evaluate creating your own chatbot using OpenAI API.

Comparing Chatbase with Botpress

If you decide to go with a 3rd party service, you might be torn between two popular platforms: Botpress and Chatbase.

Video: Do you know the best chatbot for your website? (8 min)

	GPT Integration	Customization	Pricing
Botpress	❌ Traditional style of workflow and steep learning curve	✅ Wide range of integrations	✅ Free to start
Chatbase	✅ Does everything with prompt engineering	✅ Easy customization	❌ Limited free plan options

Making the right choice

While both platforms offer unique features, Chatbase stands out as the superior choice in most instances. Here's why:

• Chatbase allows easy customization and integration with various tools
• Chatbase's user-friendly interface makes it accessible to a wide range of users. A prompt engineer can setup, tweak and improve the system. No development required
• Botpress lacks the intuitive interface of Chatbase, and is more prone non-sequeter conversation without extensive workflow development and testing

There's lots of awesome AI tools being released, but combining these can become very hard as an application scales.
Semantic Kernel can solve this problem by orchestrating all our AI services for us.

What is Semantic Kernel?

Semantic Kernel is an open source SDK developed by Microsoft for their Copilot range of AI tools.
It acts as an orchestration layer between an application and any AI services it may consume, such as the OpenAI API or Azure OpenAI, removing the need to write boilerplate code to use AI.

Microsoft - What is Semantic Kernel?
Semantic Kernel - GitHub Repo

Why use Semantic Kernel?

Semantic Kernel offers many benefits over manually setting up your AI services.

• Common AI abstractions

  • Resistant to API changes
  • Services can be easily swapped (i.e. from Azure OpenAI to OpenAI API or vice versa)
• Faster development time
• Easier maintenance

Using Semantic Kernel, it's easy to set up a basic console chat bot in under 15 lines of code!

using Microsoft.SemanticKernel;

const string endpoint = Environment.GetEnvironmentVariable("AZUREOPENAI_ENDPOINT")!;
const string key = Environment.GetEnvironmentVariable("AZUREOPENAI_API_KEY")!;
const string model = "GPT35Turbo";

var kernel = Kernel.Builder
    .WithAzureChatCompletionService(model, endpoint, key)
    .Build();

while (true)
{
    Console.WriteLine("Question: ");
    Console.WriteLine(await kernel.InvokeSemanticFunctionAsync(Console.ReadLine()!, maxTokens: 2000));
    Console.WriteLine();
}

For a more in depth walkthrough, please see Stephen Toub's article.

Comparing and classifying text can be a very time consuming process. Did you know you can use embeddings to streamline this?

What are embeddings?

Embeddings are way of grouping how similar certain pieces of text are to eachother, and take the form of a vector.
You can think of an embedding as similar to a point in 2D space with an X and Y coordinate.

The only difference is that they actually have far more dimensions. For example, embeddings generated using OpenAI's embedding models have 1536 dimensions per vector!

What can embeddings be used for?

Document Clustering
Using embeddings you can group documents based on their content without the need to manually read and classify them.

Search
Embeddings can speed up searches by a huge amount given that you can search using the vector value as opposed to text.
A good example of this is the SSW RulesGPT bot, which embeds the message you send it and uses the resulting vector to search for rules relevant to your question.

Recommendations
Embedded text can be easily compared based on it's content, making it perfect for recommending things like similar articles or books without the need to manually tag or categorise.

Cross Lingual Tasks
When you embed a piece of text the resulting embedding represents the meaning of the text and is not tied to any particular language.
This means you can use embeddings to compare different language texts without needing to read or translate the text!

How can embeddings be used?

When you have an embedding for two pieces of text you can perform a mathematical operation called cosine similarity, which measures the distance between the two vectors. The closer they are, the more similar the text.

Many databases such as Cosmos DB, Redis and Pinecone have inbuilt cosine similarity functions, making it easy to quickly compare embeddings.Other databases such as Postgres have plugins to handle vectors.

How do we get embeddings?

OpenAI provides specialised embedding models that are accessible via an API, similar to the GPT API. These models are generally cheaper than the GPT models, meaning large amounts of text can be embedded cheaply.You can find more information on these models and how to access them here.

There are also open source and self hosted models available at Hugging Face.

When you're building a custom AI application using a GPT API you'll probably want the model to respond in a way that fits your application or company.You can achieve this using the system prompt.

What is the system prompt?

Requests to and from a GPT API generally have three types of messages, also known as roles or prompts.

User
User messages are any messages that your application has sent to the model.

Assistant
Assistant messages are any messages that the model has sent back to your application.

System
The system prompt is sent with every request to the API and instructs the model how it should respond to each request. When we don't set a system prompt the user can tell the model to act however they would like it to:

For more information on system prompts, see OpenAI's documentation, or use their playground to start testing your own!