Skip to content
43

Awesome Multimodal Research

A curated list of Multimodal Related Research.

1.4k stars147 forks35 entriesLast push Aug 5, 2023 (3 years ago)License MIT

This page lists names, links and short descriptions. The original list on GitHub is the source and belongs to its authors.

News

[03/2023] OpenAI: ChatGPT plugins are tools designed specifically for language models with safety as a core principle,…

:ChatGPT plugins allow ChatGPT to access up-to-date information, run computations, or use third-party services.

In 3 listsLink may be dead

[03/2023] Google Research: Bard is an early experiment that lets you collaborate with generative AI, powered by a…

Link may be dead

[03/2023] OpenAI: GPT-4 is a large multimodal model (accepting image and text inputs, emitting text outputs) that,…

Announcement of GPT-4, a large multimodal model. OpenAI blog, March 14, 2023.

In 5 listsLink may be dead

[03/2023] Google Research: PaLM-E is a new generalist robotics model that overcomes these issues by transferring…

[03/2023] OpenAI: ChatGPT and Whisper APIs, developers can now integrate ChatGPT and Whisper models into their apps…

Link may be dead

[02/2023] MSR: Kosmos-1 is a multimodal large language model (MLLM) that is capable of perceiving multimodal input,…

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

In 4 lists

[01/2023] Google Research: 2022 & beyond: Language, vision and generative models, a post of a series in which…

[11/2022] OpenAI: ChatGPT is a sibling model to InstructGPT, which is trained to follow an instruction in a prompt and…

Announcement of ChatGPT, a conversational model trained to answer follow-up questions, admit mistakes, challenge incorrect premises, and reject inappropriate requests. OpenAI blog, November 30, 2022.

In 8 listsLink may be dead

[04/2022] OpenAI: DALL·E 2 is a new AI system that can create realistic images and art from a description in natural…

DALL·E 2 by OpenAI is a new AI system that can create realistic images and art from a description in natural language.

In 3 listsLink may be dead

[05/2021] Google: MuM, a new AI milestone for understanding information.

by Google.

In 2 lists

[03/2021] OpenAI: Multimodal Neurons in Artificial Neural Networks, which may explain CLIP’s accuracy in classifying…

Link may be dead

[01/2021] OpenAI: CLIP maps images into categories described in text, and DALL-E creates new images from text. A step…

Link may be dead

Recent Workshop

Social Intelligence in Humans and Robots

, ICRA 2021

LANTERN 2021

The Third Workshop Beyond Vision and LANguage: inTEgrating Real-world kNowledge, EACL 2021

Link may be dead

Advances in Language and Vision Research (ALVR)

, NAACL 2021

Visually Grounded Interaction and Language (ViGIL)

, NAACL 2021

Wordplay: When Language Meets Games

, NeurIPS 2020

NLP Beyond Text

, EMNLP 2020

International Challenge on Compositional and Multimodal Perception

, ECCV 2020

Link may be dead

Multimodal Video Analysis Workshop and Moments in Time Challenge

, ECCV 2020

Video Turing Test: Toward Human-Level Video Story Understanding

, ECCV 2020

Grand Challenge and Workshop on Human Multimodal Language

, ACL 2020

Link may be dead

Workshop on Multimodal Learning

, CVPR 2020

Language & Vision with applications to Video Understanding

, CVPR 2020

International Challenge on Activity Recognition (ActivityNet)

, CVPR 2020

The End-of-End-to-End A Video Understanding Pentathlon

, CVPR 2020

Towards Human-Centric Image/Video Synthesis, and the 4th Look Into Person (LIP) Challenge

, CVPR 2020

Visual Question Answering and Dialog

, CVPR 2020

Recent Tutorial

Tutorials on Multimodal Machine Learning

, CVPR 2022 && NAACL 2022

Multi-modal Information Extraction from Text, Semi-structured, and Tabular Data on the Web (Cutting-edge)

, ACL 2020

Achieving Common Ground in Multi-modal Dialogue (Cutting-edge)

, ACL 2020

Recent Advances in Vision-and-Language Research

, CVPR 2020

Neuro-Symbolic Visual Reasoning and Program Synthesis

, CVPR 2020

Large Scale Holistic Video Understanding

, CVPR 2020

A Comprehensive Tutorial on Video Modeling

, CVPR 2020

See category
99

Awesome DeepSeek Harness (DSH) Plugin

awesome-dsh-plugin/awesome-dsh-plugin

A curated list of plugins for DeepSeek Harness (dsh) · DeepSeek Harness 插件精选列表

Fresh★ 25k1416 entriesPushed today
98

Awesome OpenClaw Skills

VoltAgent/awesome-openclaw-skills

The awesome collection of OpenClaw skills. 5,400+ skills filtered and categorized from the official OpenClaw Skills Registry.🦞

Fresh★ 54k843 entriesPushed 16 days ago
96

Awesome Prompts

ai-boost/awesome-prompts

Curated prompts, frameworks and papers for LLMs and agents, with an engineering bias. Prompt engineering, context engineering, agent harnesses, red teaming.

Fresh★ 9.3k365 entriesPushed today
96

Awesome Open Source Alternatives

piotrkulpinski/open-source-alternatives

Curated list of open source alternatives to proprietary software.

Fresh★ 6.9k860 entriesPushed today
95

Awesome README

matiassingers/awesome-readme

A curated list of awesome READMEs

Fresh★ 22k144 entriesPushed 5 days ago
94

Awesome Persona Distill Skills

xixu-me/awesome-persona-distill-skills

Curated list of Agent Skills centered on people, relationships, commemorative scenes, and methodological perspectives

Fresh★ 4.7k72 entriesPushed today