Mechanical properties of a nanomechanical resonator significantly impact the performance of a resonant Nano-electromechanical system (NEMS) device. We study the mechanical properties of suspended membranes fabricated out of low-pressure chemical vapor deposited silicon nitride thin films. We fabricated doubly-clamped membranes of silicon nitride with thickness less than 50 nm and length varying from 5 to 60 μm. The elastic modulus and stress in the suspended membranes were measured using Atomic Force Microscope (AFM)-based nanomechanical spectroscopy. The elastic moduli of the suspended membranes are significantly higher than those of corresponding on-substrate thin films. We observed a reduction in net stress after the fabrication of suspended membrane, which is explained by estimating the thermal stress and intrinsic stress. We also use a mathematical model to study the stress and thickness-dependent elastic modulus of the ultrathin membranes. Lastly, we study the capillary force-gradient between the SiNx suspended membrane-Si substrate that could collapse the suspended membrane.