Who am I attacking?

The icon in the upper right-hand corner takes you random towns of varying levels that you can attack. Other people, who are not your neighbors, can also attack you.