From what I can tell it is based only your changes in diplomacy status. e.g. If you declare war it will drop a lot. But the status of a country you change status with will affect how that change affects your popularity. In other words, be nice to the popular kids and bully the unpopular.