ESC
On this page
Features > Replicate Integration
Deploying to Replicate is easy, but a bit more involved than Runpod.
Check out the AI Services tutorial to learn how to deploy your custom AI models on Replicate.
After deploying your model get your model version, its input type and modify api/ai/replicate/route.ts to make requests to your model inside your NextStarter project.
Since Replicate API is async, meaning the response is not instantaneous, we need to wait and retry if necessary. Don't worry,We handle all the waiting/retrying logic for you.
api/ai/runpod/route.ts
1
2 let data = JSON.stringify({
3 // MODELS INPUT AS A JSON OBJECT.
4 version: "#YOUR_DEPLOYED_MODEL_VERSION", // YOUR DEPLOYED MODEL VERSION,
5 input: {
6 image: base64,// Modify the input type to match the request body of your API, exact types depend on your model
7 prompt: requestBody.prompt,
8 seed: requestBody.seed,
9 style: requestBody.style,
10 },
11 });
12
13 let config = {
14 method: "post",
15 maxBodyLength: Infinity,
16 url: "https://api.replicate.com/v1/predictions",
17 headers: {
18 Authorization: `Bearer ${process.env.REPLICATE_API_TOKEN}`, // place in your API key here or use environment variables
19 "Content-Type": "application/json",
20 },
21 data: data,
22 };
23
24 // ...
25 // Since replicate api is async, we need to wait and retry if necessary.
26 try {
27 axiosRetry(axios, {
28 retryDelay: (retryCount) => {
29 return retryCount * 1;
30 },
31 retries: 15,
32 });
33
34 let result = await axios.request(config);
35 if (result.status === 200) {
36 if (result.data.status === "failed") {
37 return NextResponse.json(result.data, { status: 500 });
38 }
39
40 let waitCount = 0;
41 let predId = result.data.id;
42 while (
43 result.data.status == "starting" ||
44 result.data.status == "processing"
45 ) {
46 if (waitCount > 20) {
47 // at least 20 seconds
48 return NextResponse.json(result.data, { status: 500 }); // max wait, timeout
49 }
50 await new Promise((resolve) => setTimeout(resolve, 1000)); // wait 1 second
51 result = await axios.get(`https://api.replicate.com/v1/predictions/${predId}`, {
52 {
53 headers: {
54 Authorization: `Bearer ${process.env.REPLICATE_API_TOKEN}`,
55 "Content-Type": "application/json",
56 },
57 }
58 );
59 waitCount++;
60 if (result.data.status === "failed") {
61 return NextResponse.json(result.data, { status: 500 });
62 }
63 if (result.data.status === "succeeded") {
64 return NextResponse.json(result.data, { status: 200 });
65 }
66 }
67 } else {
68 return NextResponse.json(result.data, { status: result.status });
69 }
70 } catch (error) {
71 return NextResponse.json(error.response.data, {
72 status: error.response.status,
73 });
74 }
75}
76
77
78
79 Input Type
The input schema depends on what model you are running. To see the available inputs, click the “API” tab on the model you are running on Replicate.Modify the input type to match the request body of your API