You may blame high gas prices on rich oil company executives or greedy gas station owners. The truth is that governments rake in a larger profit at the pump than anyone—and with gas taxes on the rise in many parts of the country, there's no relief in sight. The price of a gallon of gas is based on the combination of four costs: that of crude oil, of refining gas, of distribution and marketing, and of taxes.

Crude oil costs make up about 76% of the cost of gasoline, according to U.S. Energy Information Administration (EIA). Thus $2.66 of a $3.50 gallon of gasoline is set before the oil is even refined.

Refining oil is the next step in the process—and the next expense for drivers. Gasoline is extracted from crude oil and additives, including lubricants and detergents to reduce engine deposits, are added. As of January 2012, the EIA found that refining was responsible for 6% of the cost of gasoline.

Distribution and marketing—the part of the process most apparent to consumers—constitutes another 6% of gas prices

The remaining 12%—or almost 50 cents per gallon today—goes directly to federal, state and local governments in an array of sales and excise taxes. The federal gas tax is 18.4 cents on every gallon of gasoline sold in America. State gas-tax rates vary from a low of eight cents per gallon in Alaska to a jarring 49 cents per gallon in New York. Other states where it's steep to fill up include California and Connecticut—each with 48.6-cent-per-gallon gas taxes—and Hawaii, at 47.1 cents per gallon. Some local governments have gotten in on the act, too. In California, local sales and excise taxes on gasoline average 3.1%, according to the Los Angeles Times. That works out to about 12 cents in local taxes for each gallon of gas, based on the state's current average of $3.80 per gallon.

Put this all together, and government makes far more from gas sales than all of the oil companies put together. Exxon, for example, made only seven cents per gallon of gasoline in 2011. That's a drop in the bucket compared to the nearly 50 cents per gallon that federal, state and local governments rake in on an average gallon of gas pumped in the U.S.