If you consider a mug to consist of two parts, the handle and the cup, the cup has broken into pieces and the handle is holding two of them together. So I think "two pieces held together by the handle" is the most correct way to describe it.