StarLandAI DePIN’s Computing Power Scheduling

starland.ai

The rapid development of artificial intelligence has ushered us into an era that demands extensive computing power. The advancement of AI undoubtedly requires resources from GPU-centric centers, as the training of models, especially large ones, demands significant resources. However, compared to the limited number of dedicated computing centers, there is an abundance of idle devices, including idle servers, personal PCs, and other smart terminals. These computing resources can all be utilized to contribute to AI applications, such as model inference. StarLand DePIN aims to aggregate and utilize various types of idle computing resources to perform inference tasks for different models, fully leveraging all available resources.

1. The types of devices supported by the platform

Currently, StarLand DePIN supports terminals with various operating systems, including Linux, Windows, and macOS. Therefore, your devices can easily connect to our network without requiring extensive adaptation work. Additionally, whether your device is equipped with NVIDIA or AMD GPU cards, its computing power can be effectively utilized. Of course, the computing power of devices may vary depending on factors such as the GPU model, CPU model, and memory size. Each device may contribute different amounts of computing power.

Similarly, our network supports access from mobile devices. If you have an iPhone or an Android smartphone, you can also connect to our platform. However, due to the limited computing power of mobile devices, we can only support running smaller models on them.

The Nvidia GPUs supported by us are as follows:

The AMD GPUs supported by us are as follows:

Regarding supported CPUs, we don’t have any specific requirements. We can support mainstream CPU chips from Apple, Intel, and AMD.

2. Regarding task computing and power scheduling

2.1 Computing power access

2.1.1 Computing power evaluation

To provide a reliable and scalable computing power infrastructure, StarLand combines secure hardware with cryptographic algorithms to implement a comprehensive hardware authentication and computing proof mechanism. The overall process is as follows:

1. During the startup phase of the computing power node, device self-check is performed to examine the status of GPU, CPU, and driver versions.
2. The computing power node daemon verifies the hash of the StarLand runtime image.
3. The computing power node daemon starts the StarLand runtime. If a trusted execution environment is available, the runtime will be launched based on it.
4. The StarLand runtime performs model consistency checks and loading.
5. The initialized StarLand runtime checks its runtime environment, loads the model, identity certificate, and device information, generates a runtime authentication report, and sends it to the StarLand DePIN Master in the form of heartbeats.
6. The StarLand DePIN Master verifies the runtime information received in the report and completes the node access.
7. For a single computing power evaluation inference task, the StarLand DePIN Master encrypts the task parameters and task challenge values using the runtime’s public key for distribution.
8. The runtime decrypts the task information, generates runtime challenge responses and model-specific invocation challenge values, and invokes the model to obtain inference results.
9. The runtime verifies the model validation challenge response values and inference results. Based on the challenge responses generated in step 8, it constructs a single-call computation proof and returns it to the StarLand DePIN Master. Upon receiving the response, the StarLand DePIN Master completes the inspection and results, finalizing the entire process.

Through this mechanism, we can verify whether the connected devices meet the platform requirements and assess the computing power of the accessed devices based on the execution of evaluation tasks.

2.1.2 Model deployment

The capacity to host different model sizes varies across devices. Servers equipped with professional-grade GPUs naturally possess stronger computing power, allowing them to support larger model sizes. Personal PCs with gaming-grade GPUs may have slightly lower computing power but can still support some medium-sized models. Smart terminals equipped only with CPUs, without GPUs, may support smaller model sizes.

StarLand DePIN selects suitable models for deployment on your connected devices based on the results of computing power evaluations. The models we choose to deploy will utilize a portion of your device’s computing power, leaving some resources available for other activities. For example, if your device is equipped with a professional-grade GPU like the A100, we may deploy larger models like llama3 70b, which have more model parameters and often result in better and more precise inference. Conversely, if your device has a consumer-grade GPU like the GTX 3060, we may deploy smaller models like phi3 3.8b, which require fewer resources. This approach ensures that device performance is effectively utilized without wasting computing resources and that device computing resources are not overly consumed, thereby ensuring the normal operation of daily tasks.

2.2 Computing power matching

StarLand DePIN connects devices from around the world, and it also handles inference tasks from various global sources. Some devices have high computing power, while others have lower computing power. Additionally, some connected devices may have high idle capacity, while others may be heavily loaded. StarLand DePIN comprehensively considers these factors when allocating inference tasks.

• Model Matching for Inference Tasks

As previously mentioned, we deploy different models based on the varying computing power of devices. Different inference tasks received by the platform have different requirements for models. Some tasks require larger models, relying on higher computing power, while others can be accomplished with smaller models without significant computing power demands. The platform matches the models deployed on all connected devices according to the model requirements of different tasks. It selects devices with matched models to complete the corresponding inference tasks, ensuring the high-quality completion of inference tasks.

• Load Matching

Even if the same model is deployed and can handle similar inference task requirements, the load on different devices may vary. The platform considers the load situation of devices that can complete inference tasks and assigns newly received tasks to relatively idle devices for inference. For example, between a device with 60% load and another with 30% load, we would prioritize the latter for inference.

• Geographic Matching

We know that cross-geographical network communication incurs significant overhead and higher latency. To efficiently complete inference tasks, we prioritize assigning received inference tasks to devices in the same geographical region for inference. For instance, if two devices with similar loads and deployed with the same model are available, one in Asia and the other in the Americas, when the platform receives an inference task matched to this model from Asia, it will be prioritized for assignment to the device in Asia.

By considering model requirements, load situations, and geographical locations, StarLand DePIN intelligently distributes inference tasks to connected devices.

2.3 Handling Device Malfunctions

StarLand DePIN conducts regular health checks on all connected devices to ensure that tasks are assigned to healthy devices. If a device fails the health check, the platform marks it as inactive for the current check cycle and records the event. Devices that frequently experience failures have their task priority lowered. During the next health check, if a previously faulty device is found to be functioning properly, the platform reactivates it for task assignment.

For inference tasks running on faulty devices that experience timeouts or failures to complete, the platform reallocates these tasks to healthy devices based on the scheduling rules described above.

The health check process follows the flow outlined below:

1. Verify the device’s identity to determine its current owner.
2. Check the device’s network status, including connectivity and bandwidth.
3. Monitor the device’s CPU status.
4. Examine the device’s GPU status and driver information.
5. Assess the current workload on the device.

Compute Power Offline

Devices connected to the platform by computing power providers can choose to go offline voluntarily. When a computing power provider decides not to take on inference tasks, they can terminate their service by issuing an exit command through the computing power node daemon. Additionally, the platform maintains a record of faulty computing devices and may blacklist devices that experience frequent failures, enforcing mandatory offline status for them.