Github Models - Free AI Model Experience and Evaluation Platform

Github Models is a new AI-related service introduced by Github. From its documentation (Find and experiment with AI models for free), it can be understood that this is a "free" platform for experiencing and evaluating large language models.

As shown in the figure below, Github Models provides a series of mainstream large language models, including gpt-4o, gpt-4o-mini, llama-3.1 405b, phi-3.5-mini-128k, and so on. You can directly interact with the models on the platform's Playground, or experience them programmatically using the OpenAI or Azure AI Inference SDK.

The platform sets three types of limitations for AI models of different specifications: Low, High, and Embedding. Different types of users have different resource limitations. The limitations are mainly based on four dimensions: requests per minute, requests per day, input and output token count per request, and concurrent requests, in order to prevent resource abuse.

As of August 30, 2024, the resource limitation rules for Github Models are shown in the figure below:

Currently, Github Models is still in public testing phase. You can visit https://github.com/marketplace/models to apply to join the waiting list. On the homepage of the Github user center, you can see the image of the Github Models public testing application. Click on it to apply.

Usage Example

Once your application is approved, you can return to the Marketplace to select the AI model you want to experience.

Taking Meta-Llama-3.1-405B-Instruct as an example, on the model page, you can see the introduction information of the model. It is recommended to pay attention to the Rate limit tier resource limitation level on the right sidebar.

Click on Playground to directly interact with the model online.

Switch to the Code tab to view the underlying code of the current conversation. These codes can be directly reused locally.

You can also click the Getting started button in the upper right corner of the page to view the usage methods of other languages and SDKs.

If you need to call the AI models provided by Github Models locally through the SDK, you need to apply for a Personal Access Token. Only the most basic "public repository read-only permission" is required, and no other permissions need to be assigned. After generating the token, you can use it to access the AI models on the platform through the SDK.

Summary

From the introduction of Github Models, it is not difficult to see that this is a transitional service for Azure commercial products. By attracting developers on Github to experience and evaluate AI models, it aims to guide a group of users with needs and potential to become commercial users of Azure AI.