I was able to find a source from The Metropolitan Museum of Art’s website. it seems that it would’ve actually gone up to the 20th letter.

A number of polyhedral dice made in various materials have survived from the Hellenistic and Roman periods, usually from ancient Egypt when known. Several are in the Egyptian or Greek and Roman collections at the Museum. The icosahedron – 20-sided polyhedron – is frequent. Most often each face of the die is inscribed with a number in Greek and/or Latin up to the number of faces on the polyhedron.