Sep 19, 2023 · In this work, we introduce the Qwen-VL series, a set of large-scale vision-language models (LVLMs) designed to perceive and understand both texts and images. …

The overall network architecture of Qwen-VL consists of three components and the details of model parameters are shown in Table 1: Large Language Model: Qwen-VL adopts a large …

Qwen-1.5-7B Qwen-1.5-32B Qwen-1.5-72B Claude-3.5-sonnet Gemma-2-9b Gemma-2-27b Llama-3.1-8B Llama-3.1-70B Llama-3.1-405B Safe Over-Refusal Claude Gemini GPT-3.5 …

Published as a conference paper at ICLR 2024 3.1 LOSS OF HIGH FREQUENCY INFORMATION - "NTK-AWARE" INTERPOLATION If we look at rotary position embeddings …

Sep 12, 2024 · Our experiments demonstrate AIR's effectiveness across state-of-the-art LLMs, achieving an attack success rate (ASR) exceeding $\textbf{90}$% on most models, including …

Feb 1, 2023 · We propsoe MARVEL to adaptively allocate the parameter budget among weight matrices in correspondence to their importance.

Sep 26, 2024 · Additionally, we evaluate three categories of LLMs against this benchmark: 1) proprietary LLM services such as GPT-4; 2) public LLMs like QWen; and 3) specialized …

Welcome to the OpenReview homepage for ICLR 2024

Building upon this formulation, the experts in both Qwen (Team, 2024) and Mixtral (Jiang et al., 2024) adopt the structure of LLaMA (Touvron et al., 2023). Specifically, the feed-forward …

Apr 24, 2023 · Recent years have witnessed a big convergence of language, vision, and multi-modal pretraining. In this work, we present mPLUG-2, a new unified paradigm with …