Establishes the overall multi-layer framework for AIGC systems, standardizing the implementation and evaluation workflows for cross-modal content generation applications, audio generation, code-assisted generation, and Retrieval-Augmented Generation (RAG) systems.
F.748.56 General framework of the AIGC system
Single modality (unimodal) tasks include implementing the generation of large amounts of data with similar characteristics based on existing data transforming the characteristics or modality of existing data based on a demand. Taking image generation as an example, the training data for a generative model is a single modality. The inputs are the images to be generated or transformed, and the task could be generating a large number of images of similar style based on a specific style of image or transforming a photograph into a certain painting style. The workflow of such a single modality task is shown below.
F.748.56 Workflow of a single modality task
A multi-modality (multimodal) task generally refers to generating multimodal data with corresponding characteristics based on an input demand. Training data for the generative model contains each modality and its description. The inputs are prompts, such as text-generated images or text-generated audio prompts. Tasks could be generating images or audio based on the input text to generate a corresponding image or audio, or a description based on the image. The workflow of such a multi-modality task is shown below.
F.748.56 Workflow of a multi-modality task F.748.45 Structure of the requirement and evaluation methods of code generation based on the foundation model
F.748.52 Framework of requirements and evaluation methods for retrieval augmented generation of large scale pre-trained model
6.2 Media Processing & Enhancement
Standardizes the business processing workflows for refining semantic editing, interpolating and restoring old films, noise reduction, and converting unstructured audio/video into structured data relying on multimodal AI models.
