RewardSDS: Aligning Score Distillation via Reward-Weighted Sampling

Itay Chachy, Guy Yariv, Sagie Benaim
The Hebrew University of Jerusalem

Paper arXiv Code

Text-to-3D

"A penguin with a brown bag in the snow"
"A man with a red scarf"
"A shitzu with a bowtie"
"Two dogs in the park"
"A DSLR photo of a ghost eating a hamburger"
"A basketball player dunking a basketball"

Abstract

Score Distillation Sampling (SDS) has emerged as a highly effective technique for leveraging 2D diffusion priors for a diverse set of tasks such as text-to-3D generation. While powerful, SDS still struggles with achieving fine-grained alignment to user intent. To overcome this limitation, we introduce RewardSDS, a novel approach that weights noise samples based on the alignment scores of a reward model, producing weighted SDS loss. This loss prioritizes gradients from noise samples that yield aligned high-reward output. Our approach is broadly applicable and can be applied to diverse methods extending SDS. In particular, we also demonstrate its applicability to Variational Score Distillation (VSD) by introducing RewardVSD. We evaluate RewardSDS and RewardVSD on text-to-image, 2D editing, and text-to-3D generation tasks, demonstrating a significant improvement over SDS and VSD on a diverse set of metrics measuring generation quality and alignment to desired reward models, enabling state-of-the-art performance.


Method

Method Figure

Overview of our method, an image is first rendered from a given view and N random noises are applied (at a given timestep). The noisy images are then scored by denoising them and applying a reward model on the output. These scores are then mapped to corresponding weights, which are used to weigh the contribution of each noisy sample in score distillation.


RewardSDS vs MVDream

NeRF-Based Scenes

"A cartoon cat eating a cheesecake"
Reward SDS
MVDream
"A penguin with a brown bag in the snow"
Reward SDS
MVDream
"A man with a beard, wearing a suit, holding a pink briefcase"
Reward SDS
MVDream
"A bulldog wearing a black pirate hat"
Reward SDS
MVDream

3DGS-Based Scenes

"Two dogs in the park"
Reward SDS
MVDream
"Two computer monitors on a brown wooden table"
Reward SDS
MVDream
"A DSLR photo of the Imperial State Crown of England"
Reward SDS
MVDream
"A banana on the left of an apple"
Reward SDS
MVDream

Reward Model Effect

"A skyscraper that reaches the clouds"
Aesthetic
ImageReward
"Mini Chinatown"
Aesthetic
ImageReward

BibTeX

@misc{chachy2025rewardsdsaligningscoredistillation,
      title={RewardSDS: Aligning Score Distillation via Reward-Weighted Sampling}, 
      author={Itay Chachy and Guy Yariv and Sagie Benaim},
      year={2025},
      eprint={2503.09601},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2503.09601}, 
}