Based on the study of the solids conveying in the vane extruder, a new concept of the differential shear-compression theory is presented. The objective existing of shear and compressive forces is proved by the mathematical method. And the exerted force is proved to be depended on the compression displacement and the differential of compression speed when solid materials are transported in the solids conveying zone of the vane extruder at any moment. Because of the rotor eccentricity to the stator, a conveying mathematical model between the thickness of the material differential laminate and the rotor angle is established. The nonlinear function E(β) andG(β), related to the thickness, density and compactness of materials are obtained through experiment, and the radial force of inner surface of the stator acted on the differential laminate is calculated by the Hooke's law. Based on the torque balance of the material laminate, the force is calculated by integration in the chamber. The finite element analysis of the mathematic model are applied to verify the validity of the differential shear-compression theory, which is also practical to metal material rolling, high pressure double roll crush and other processes with converging geometry.