Ad Server is the server that does the ad serving. It can belong to the publisher or to the advertiser. In Supply (Publisher) and in Demand (Advertiser), the Adserver does the same. For both, the adserver will measure the impressions, clicks, errors. If connected to RTB, the matches, bid requests, and other metrics.
Altought they are pretty similar, normally the ad tech companies like Google create two different solutions. Doubleclick Digital Marketing for Advertisers, where they include Doubleclick Rich Media, Doubleclick Bid Manager, Doubleclick Campaign Manager, Doubleclick Studio. Then, for publishers, Google offer a solution for publishers called DFP – Doubleclick for Publishers. All this solutions are connected then via RTB trough Doubleclick Ad Exchange, or trought another Ad Exchanges, SSP´s or manually via tag implementation.
First Adserver was created back in 1995, by a US company called FocaLink Media Services, today named AdKnowledge, after being purchased.