Sorry about the problem you are experiencing. From the sounds of it the board is not getting enough power without USB. Could you please double check the pins on the APM board where you connect the power module (as many times they can be bent). They are very small so might be good idea to use magnifying glass or something to see properly).
Also could you also check the power module cable and pins to ensure there is a good connection on that too.
Otherwise if you connect the JP1 (and remove power module) and power your APM via one of your ESC’s does that work?