Consider $mxy^{3}+2my^{2}-mxy-2m$. Factor out $m$.
$$m\left(xy^{3}+2y^{2}-xy-2\right)$$
Consider $xy^{3}+2y^{2}-xy-2$. Do the grouping $xy^{3}+2y^{2}-xy-2=\left(xy^{3}+2y^{2}\right)+\left(-xy-2\right)$, and factor out $y^{2}$ in the first and $-1$ in the second group.
$$y^{2}\left(xy+2\right)-\left(xy+2\right)$$
Factor out common term $xy+2$ by using distributive property.
$$\left(xy+2\right)\left(y^{2}-1\right)$$
Consider $y^{2}-1$. Rewrite $y^{2}-1$ as $y^{2}-1^{2}$. The difference of squares can be factored using the rule: $a^{2}-b^{2}=\left(a-b\right)\left(a+b\right)$.
