Model Details

This model is a mixed gguf q2ks format of Qwen/Qwen3.5-35B-A3B generated by intel/auto-round algorithm. Embedding layer and lm-head layer are fallback to 8 bits and non expert layers are fallback to 4 bits. Please refer to Section Generate the model for more details.

Please follow the license of the original model.

How To Use

Llama.cpp inference.

wget https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg
./llama-cli  -m Qwen3.5-35B-A3B-w2g16/Qwen3.5-35B-A3B-Q2_K_S.gguf  -mm Qwen3.5-35B-A3B-w2g16/mmproj-model.gguf

> /image demo.jpeg

Loaded media from 'demo.jpeg'

> Describe this image.

[Start thinking]

The user wants a description of the image.

1.  **Identify the main subjects:** A woman and a dog (looks like a yellow Labrador Retriever).
2.  **Identify the setting:** A sandy beach with the ocean in the background. The lighting suggests sunset or sunrise (golden hour).
3.  **Describe the action:** The woman is sitting on the sand. The dog is sitting next to her. The dog is reaching out its left paw and placing it on the woman's hand. The woman is smiling and looking at the dog.
4.  **Describe the details:**
    *   **The Woman:** Long dark hair, wearing a plaid shirt (blue/white/black), dark pants (maybe jeans or capris), barefoot. She has a white watch on her wrist.
    *   **The Dog:** Light tan/yellow fur, wearing a harness with a colorful pattern (blue with colorful dots/flowers). It has a leash attached.
    *   **The Background:** The ocean is calm with small waves breaking. The sky is bright, almost washed out, indicating strong backlighting from the sun.
5.  **Synthesize into a coherent description:** Start with the overall scene, then details about the subjects, their interaction, and the atmosphere.

*Drafting the description:*
The image shows a heartwarming scene on a sandy beach during what looks like late afternoon or sunset. A young woman with long dark hair is sitting on the sand, facing right. She is wearing a blue and white plaid shirt and dark pants. She is smiling warmly. Next to her is a yellow Labrador Retriever sitting on its hind legs. The dog is wearing a blue patterned harness. The dog is extending its left paw and resting it gently on the woman's hand, which she is holding up. The woman seems to be interacting with the dog, possibly giving it a treat or just playing a "shake" trick. The background features a calm ocean with gentle waves rolling in. The sun is bright in the upper right, creating a lens flare and casting a warm, golden glow over the scene. The sand is textured with footprints and paw prints.

*Refining:* Make it flow better. Mention the "handshake" interaction clearly. Note the leash on the sand.

*Final Polish:*
This image captures a sweet moment between a woman and her dog on a beach at sunset.
- **Subjects:** A woman with long dark hair, wearing a plaid shirt and dark pants, sits on the sand. Beside her is a yellow Labrador Retriever wearing a patterned harness.
- **Action:** The dog is sitting upright and has extended its left paw to rest on the woman's hand. The woman is smiling and looking at the dog, holding its paw. It looks like they are doing a "shake" trick or sharing a gentle moment.
- **Setting:** They are on a wide sandy beach. The ocean is in the background with small waves breaking. The lighting is very bright and warm, suggesting the sun is setting (golden hour), casting a soft glow and creating a lens flare in the top right corner.
- **Details:** The dog has a leash lying on the sand near its tail. The woman is barefoot.

This covers all the key aspects. I will present this in a structured paragraph format.

[End thinking]

This image captures a heartwarming moment between a young woman and her dog on a sandy beach, likely during the "golden hour" of sunset.

**The Subjects:**
*   **The Woman:** She is sitting on the sand, facing the dog. She has long, dark hair and is wearing a blue and white plaid button-down shirt over dark pants. She is barefoot and has a white watch on her wrist. She is smiling warmly, looking directly at the dog.
*   **The Dog:** A yellow Labrador Retriever is sitting upright next to her. It is wearing a blue harness with a colorful pattern.

**The Action:**
The dog is extending its left paw and gently placing it on the woman's hand. The woman is holding the paw, suggesting they are playing a game like "shake" or sharing a gentle moment of connection.

**The Setting:**
*   **Background:** The ocean is visible in the background with gentle waves rolling in. The horizon is slightly hazy.
*   **Lighting:** The sun is positioned in the upper right corner, creating a bright, hazy light that washes out the sky and casts a warm, golden glow over the sand and the subjects.
*   **Foreground:** The sand is textured with footprints and ripples. A red leash lies on the sand near the dog's tail.

Generate the model

Here is the sample command to reproduce the model

auto_round --model Qwen/Qwen3.5-35B-A3B --iters 0 --output_dir tmp_autoround --scheme gguf:q2_k_mixed

Ethical Considerations and Limitations

The model can produce factually incorrect output, and should not be relied on to produce factually accurate information. Because of the limitations of the pretrained model and the finetuning datasets, it is possible that this model could generate lewd, biased or otherwise offensive outputs.

Therefore, before deploying any applications of the model, developers should perform safety testing.

Caveats and Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model.

Here are a couple of useful links to learn more about Intel's AI software:

Intel Neural Compressor link

Disclaimer

The license on this model does not constitute legal advice. We are not responsible for the actions of third parties who use this model. Please consult an attorney before using this model for commercial purposes.

Cite

@article{cheng2023optimize, title={Optimize weight rounding via signed gradient descent for the quantization of llms}, author={Cheng, Wenhua and Zhang, Weiwei and Shen, Haihao and Cai, Yiyang and He, Xin and Lv, Kaokao and Liu, Yi}, journal={arXiv preprint arXiv:2309.05516}, year={2023} }

arxiv github