Found is the complete nonlinear action of multiple D$0$-brane system (mD$0$) in three dimensional type II superspace which is invariant under rigid $D=3$ ${\cal N}=2$ spacetime supersymmetry and under local worldline supersymmetry generalizing the $\kappa-$symmetry of single D$0$-brane action. We show that a particular representative of this family of actions can be obtained by dimensional reduction of the action of $D=4$ non-Abelian multiwaves (nAmW), the $D=4$ counterpart of 11D multiple M-wave (mM0) action, that we have also constructed in this paper. This reduction results in an action with is nonlinear due to the presence of a certain function ${\cal M}({\cal H})$ of the relative motion Hamiltonian ${\cal H}$, the counterpart of which enters the 4D nAmW action linearly. Curiously, the action possesses double supersymmetry also for an arbitrary function ${\cal M}({\cal H})$. In particular for ${\cal M}=const$ we find a dynamical system describing the sum of single D$0$ action and the action of 1d dimensional reduction of the $D=3$ ${\cal N}=2$ SYM coupled to the worldline supergravity induced by the embedding of the center of energy motion into the $D=3$ ${\cal N}=2$ superspace.