That's the way its supposed to be-- lighter at low speed, heavier at high speeds. Make sure your wing angle of incidence is identical and you don't have a bunch of aileron trim in to counter an out of rig wing. The only place it might be rubbing is near the speed brake pocket where the aileron push tube passes very close to the pocket -- but that is at the extreme deflection of the aileron -- not at the normal zero deflection angle found in typical cruise. Also make sure there is no binding under the instrument panel (loose wiring, etc.) on the crossover tube. Otherwise, it is aerodynamic loads and stick force combos (design issues) that protect you from over stressing the airframe at high speeds.

Jeff Edwards LIVP N619SJ