Not limited to 'email lists' members can be allowed certain (limited) site privileges. There are many member security levels. These affect access to password protected pages and also to emailing lists. They are explained in more detail here
By assigning a member record to a certain administrator and configuring the additional administrators access settings, you can set it up so that each admin can only access their assigned members records.