console.error("Error performing face swap:",error);
}
```
### Parameter Description
-`swap_image`: URL of the image containing the face to be swapped.
-`target_image`: URL of the target image where the face will be swapped.
### Notes
- Ensure that the provided image URLs are publicly accessible.
- The API may take some time to process the images, consider implementing appropriate wait or retry logic.
- Handle possible errors, such as network issues or API limitations.
### Example Response
The API response will contain the URL of the processed image or other relevant information. Parse and use the response data according to the actual API documentation.
## Advanced Usage
- Consider implementing error retry mechanisms.
- Add image validation logic to ensure the provided URLs point to valid image files.
- For production environments, consider implementing rate limiting and caching mechanisms to optimize API usage.
-`task`: String, the text content to be converted into speech.
-`task`: String, The type of task the model needs to perform,choose[`image_captioning`,`image_text_matching`],
-`image`: String, the URL of the audio file used as the voice sample for cloning.
- default: `image_captioning`
-`image_captioning`: Identify the scene in the picture and give the result
-`image_text_matching`: The result is obtained by matching the given caption parameters with the scene in the picture
-`visual_question_answering`: Ask questions about the scene in the picture
-`image`: String, provide the URL of the image that needs to be recognized.
-`caption`:String, A text description of the scene in the image
-`question`:String,Provide a textual description of the questions
### Notes
### Notes
- Ensure that the provided audio URL is publicly accessible and of good quality to achieve the best cloning effect.
- Ensure that the provided image URL is publicly accessible and of good quality to achieve the best recognition results.
- The API may take some time to process the input and generate the result, consider implementing appropriate wait or loading states.
- The API may take some time to process the input and generate the result, consider implementing appropriate wait or loading states.
- Handle possible errors, such as network issues, invalid input, or API limitations.
- Handle possible errors, such as network issues, invalid input, or API limitations.
- Adhere to the terms of use and privacy regulations, especially when handling voice samples of others.
- Adhere to the terms of use and privacy regulations, especially when handling image samples of others.
### Example Response
### Example Response
The API response will contain the URL of the generated cloned voice or other relevant information. Parse and use the response data according to the actual API documentation.
The API response will contain the results of the image recognition or other relevant information. Parse and use the response data according to the actual API documentation.
## Advanced Usage
## Advanced Usage
- Implement batch text-to-speech conversion by processing multiple text segments in a loop or concurrent requests.
- Unimodal encoders, which separately encode image and text. The image encoder is a vision transformer. The text encoder is the same as BERT. A token is appended to the beginning of the text input to summarize the sentence.
- Add a user interface that allows users to upload their own voice samples and input custom text.
- Image-grounded text encoder, which injects visual information by inserting a cross-attention layer between the self-attention layer and the feed forward network for each transformer block of the text encoder. A task-specific token is appended to the text, and the output embedding of is used as the multimodal representation of the image-text pair.
- Implement voice post-processing features, such as adjusting volume, adding background music, or applying audio effects.
- Image-grounded text decoder, which replaces the bi-directional self-attention layers in the text encoder with causal self-attention layers. A special token is used to signal the beginning of a sequence.
- Integrate a voice storage solution to save and manage the generated voice files.
- Image-Text Contrastive Loss (ITC) activates the unimodal encoder. It aims to align the feature space of the visual transformer and the text transformer by encouraging positive image-text pairs to have similar representations in contrast to the negative pairs.
- Consider implementing a voice recognition feature to convert the generated voice back to text for verification or other purposes.
- Image-Text Matching Loss (ITM) activates the image-grounded text encoder. ITM is a binary classification task, where the model is asked to predict whether an image-text pair is positive (matched) or negative (unmatched) given their multimodal feature.
- Language Modeling Loss (LM) activates the image-grounded text decoder, which aims to generate textual descriptions conditioned on the images.
"context":"question: what animal is this? answer: panda",
"question":"what country is this animal from? ",
"temperature":1
}
};
};
```
```
...
@@ -65,21 +96,22 @@ try {
...
@@ -65,21 +96,22 @@ try {
### Parameter Description
### Parameter Description
-`image`: String, the text content to be converted into speech.
-`image`: String, Input image to query or caption.
-`caption`: String, the URL of the audio file used as the voice sample for cloning.
-`caption`: Boolean, Select if you want to generate image captions instead of asking questions.
-`question`: String, specifies the language of the text, with "en" indicating English.
-`context`: String,Optional - previous questions and answers to be used as context for answering current question.
-`temperature`: Boolean, whether to perform cleanup processing on the generated voice.
-`question`: String, Question to ask about this image. Leave blank for captioning.
-`temperature`: Number, Temperature for use with nucleus sampling.
### Notes
### Notes
- Ensure that the provided audio URL is publicly accessible and of good quality to achieve the best cloning effect.
- Ensure that the provided image URL is publicly accessible and of good quality to achieve the best recognition results.
- The API may take some time to process the input and generate the result, consider implementing appropriate wait or loading states.
- The API may take some time to process the input and generate the result, consider implementing appropriate wait or loading states.
- Handle possible errors, such as network issues, invalid input, or API limitations.
- Handle possible errors, such as network issues, invalid input, or API limitations.
- Adhere to the terms of use and privacy regulations, especially when handling voice samples of others.
- Adhere to the terms of use and privacy regulations, especially when handling image samples of others.
### Example Response
### Example Response
The API response will contain the URL of the generated cloned voice or other relevant information. Parse and use the response data according to the actual API documentation.
The API response will contain the results of the image recognition or other relevant information. Parse and use the response data according to the actual API documentation.
## Advanced Usage
## Advanced Usage
...
@@ -87,5 +119,4 @@ The API response will contain the URL of the generated cloned voice or other rel
...
@@ -87,5 +119,4 @@ The API response will contain the URL of the generated cloned voice or other rel
- Add a user interface that allows users to upload their own voice samples and input custom text.
- Add a user interface that allows users to upload their own voice samples and input custom text.
- Implement voice post-processing features, such as adjusting volume, adding background music, or applying audio effects.
- Implement voice post-processing features, such as adjusting volume, adding background music, or applying audio effects.
- Integrate a voice storage solution to save and manage the generated voice files.
- Integrate a voice storage solution to save and manage the generated voice files.
- Consider implementing a voice recognition feature to convert the generated voice back to text for verification or other purposes.
- Consider implementing a voice recognition feature to convert the generated voice back to text for verification or other purposes.
@@ -36,7 +36,7 @@ const ai_options = new AIOptions({
...
@@ -36,7 +36,7 @@ const ai_options = new AIOptions({
constaonweb=newAI(ai_options);
constaonweb=newAI(ai_options);
```
```
### 3. Prepare Input Data
### 3. Prepare Input Data Example
```js
```js
constdata={
constdata={
...
@@ -53,12 +53,27 @@ const data = {
...
@@ -53,12 +53,27 @@ const data = {
};
};
```
```
```js
constdata={
input:{
"text":"chat T T S is a text to speech model designed for dialogue applications. \\n[uv_break]it supports mixed language input [uv_break]and offers multi speaker \\ncapabilities with precise control over prosodic elements [laugh]like like \\n[uv_break]laughter[laugh], [uv_break]pauses, [uv_break]and intonation. \\n[uv_break]it delivers natural and expressive speech,[uv_break]so please\\n[uv_break] use the project responsibly at your own risk.[uv_break]",
-`text`: String, the text content to be converted into speech.
-`text`: String, Text to be synthesized
-`speaker`: String, the URL of the audio file used as the voice sample for cloning.
-`top_k`: Number, Top-k sampling parameter
-`language`: String, specifies the language of the text, with "en" indicating English.
-`top_p`: Number, Top-p sampling parameter
-`cleanup_voice`: Boolean, whether to perform cleanup processing on the generated voice.
-`voice`: Number, Voice identifier
-`prompt`:String,Prompt for refining text
-`skip_refine`:Number,Skip refine text step
-`temperature`:Number,Temperature for sampling
-`custom_voice`:Number,Custom voice identifier
### Notes
### Notes
- Ensure that the provided audio URL is publicly accessible and of good quality to achieve the best cloning effect.
- Ensure that the provided text is of good quality to achieve the best inference results.
- The API may take some time to process the input and generate the result, consider implementing appropriate wait or loading states.
- The API may take some time to process the input and generate the result, consider implementing appropriate wait or loading states.
- Handle possible errors, such as network issues, invalid input, or API limitations.
- Handle possible errors, such as network issues, invalid input, or API limitations.
- Adhere to the terms of use and privacy regulations, especially when handling voice samples of others.
- Adhere to the terms of use and privacy regulations.
### Example Response
### Example Response
The API response will contain the URL of the generated cloned voice or other relevant information. Parse and use the response data according to the actual API documentation.
The API response will contain the URL of the generated text-to-speech output or other relevant information. Parse and use the response data according to the actual API documentation.
@@ -36,7 +36,7 @@ const ai_options = new AIOptions({
...
@@ -36,7 +36,7 @@ const ai_options = new AIOptions({
constaonweb=newAI(ai_options);
constaonweb=newAI(ai_options);
```
```
### 3. Prepare Input Data
### 3. Prepare Input Data Example
```js
```js
constdata={
constdata={
...
@@ -66,22 +66,22 @@ try {
...
@@ -66,22 +66,22 @@ try {
### Parameter Description
### Parameter Description
-`image`: String, the text content to be converted into speech.
-`image`: String, Please provide the image file that needs to be processed.
-`upscale`: String, the URL of the audio file used as the voice sample for cloning.
-`upscale`: String, The final upsampling scale of the image.
-`face_upsample`: String, specifies the language of the text, with "en" indicating English.
-`face_upsample`: String, Upsample restored faces for high-resolution AI-created images.
-`background_enhance`: Boolean, whether to perform cleanup processing on the generated voice.
-`background_enhance`: Boolean, Enhance background image with Real-ESRGAN.
-`codeformer_fidelity`: Boolean, whether to perform cleanup processing on the generated voice.
-`codeformer_fidelity`: Boolean, Balance the quality (lower number) and fidelity (higher number).
### Notes
### Notes
- Ensure that the provided audio URL is publicly accessible and of good quality to achieve the best cloning effect.
- Ensure that the provided image URL is publicly accessible and of good quality to achieve the best recognition results.
- The API may take some time to process the input and generate the result, consider implementing appropriate wait or loading states.
- The API may take some time to process the input and generate the result, consider implementing appropriate wait or loading states.
- Handle possible errors, such as network issues, invalid input, or API limitations.
- Handle possible errors, such as network issues, invalid input, or API limitations.
- Adhere to the terms of use and privacy regulations, especially when handling voice samples of others.
- Adhere to the terms of use and privacy regulations, especially when handling image samples of others.
### Example Response
### Example Response
The API response will contain the URL of the generated cloned voice or other relevant information. Parse and use the response data according to the actual API documentation.
The API response will contain the results of the image recognition or other relevant information. Parse and use the response data according to the actual API documentation.
-`text`: String, the text content to be converted into speech.
-`seed` Number,Provide the seeds required for model inference
-`speaker`: String, the URL of the audio file used as the voice sample for cloning.
-`image` String,Please provide the image file that needs to be processed.
-`language`: String, specifies the language of the text, with "en" indicating English.
-`scale` Number,Scale for classifier-free guidance
-`cleanup_voice`: Boolean, whether to perform cleanup processing on the generated voice.
-`steps` Number,Provide the steps required for model inference
-`prompt` String, Please provide the prompt that needs to be inferred.
-`structure` String,Controlnet structure to condition on
-`low_threshold` Number,[canny only] Line detection low threshold
-`high_threshold` Number,[canny only] Line detection high threshold
-`image_resolution` String,Resolution of output image (will be scaled to this as its smaller dimension)
-`num_outputs` Number,Number of images to output (higher values may OOM),default:1
-`eta` Number,Controls the amount of noise that is added to the input data during the denoising diffusion process. Higher value -> more noise
-`negative_prompt` String,Provide the negative prompt required for model inference
### Notes
### Notes
- Ensure that the provided audio URL is publicly accessible and of good quality to achieve the best cloning effect.
- Ensure that the provided image URL is publicly accessible and of good quality to achieve the best recognition results.
- The API may take some time to process the input and generate the result, consider implementing appropriate wait or loading states.
- The API may take some time to process the input and generate the result, consider implementing appropriate wait or loading states.
- Handle possible errors, such as network issues, invalid input, or API limitations.
- Handle possible errors, such as network issues, invalid input, or API limitations.
- Adhere to the terms of use and privacy regulations, especially when handling voice samples of others.
- Adhere to the terms of use and privacy regulations, especially when handling image samples of others.
### Example Response
### Example Response
```js
The API response will contain the results of the image recognition or other relevant information. Parse and use the response data according to the actual API documentation.
The API response will contain the URL of the generated cloned voice or other relevant information. Parse and use the response data according to the actual API documentation.