A second-order accurate and efficient Immersed Boundary Method (IBM) has been developed for simulating particle-laden flows. Recently, this method has been combined with a soft-sphere collision model to accommodate inter-particle and particle-wall collisions. Details of the collision model are given. Results are shown from a lubrication study of non-touching particles at close distance from each other. The numerical results for the drag force acting on the particles agree well with exact solutions, except when the gap width between the particles becomes significantly smaller than the numerical grid spacing. For very small gap width, lubrication force corrections are proposed for the normal approach between particles based on asymptotic analytical solutions. Results are presented from a numerical study of sphere-wall collisions in a viscous fluid. The simulated behavior of the coefficient of restitution as function of the Stokes number based on the particle impact velocity, is in good agreement with experimental data.