Rethinking Neural Thickets Without Weight Perturbations
A paper argues that Neural Thickets gains can come from output diversity, validation-based selection, and voting, without perturbing model weights. It tests this approach on reasoning and vision tasks.
The authors reproduce Neural Thickets gains but find that full-weight perturbations are not required. A frozen model with random output-logit biases matches or exceeds full-weight perturbations on most reasoning tasks. They call the broader procedure **Perturb-Select-Vote (PSV)**.
For inference pipelines, this points to testing sampling, validation-based selection, and voting before adding weight perturbations. The paper also reports that logit PSV improves performance under vision domain shift, including Earth observation.
The authors reproduce Neural Thickets gains but find that full-weight perturbations are not required. A frozen model with random output-logit biases matches or exceeds full-weight perturbations on most reasoning tasks. They call the broader procedure **Perturb-Select-Vote (PSV)**. For inference pipelines, this points to testing sampling, validation-based selection, and voting before adding weight perturbations. The paper also reports that logit PSV improves performance under vision domain shift, including Earth observation. The results are reported on the tasks studied; they do not establish that PSV is better for every model or workload. The proposed explanation is that diversity and selection account for gains previously attributed to nearby weight-space experts.
PSV shifts the explanation for Neural Thickets gains from weight-space perturbation toward sampling, validation-based selection, and voting, including in the reported vision domain-shift tasks. This makes simpler frozen-model inference pipelines worth testing, but the supplied results support only the studied models and workloads.