The Local Government Association says councils are being forced to subsidise free travel at the cost of other essential services.
Free bus services for the elderly and disabled people are under threat as councils struggle to fund the services.
Funding for free, off-peak travel has fallen by 39% since 2010, according to the Local Government Association.
Councils are now being forced to subsidise free travel at the cost of other essential services, the LGA added.
Among the county councils forced to cut back on the services due to funding shortages are Buckinghamshire, Cumbria and Somerset.
Peter Box, the LGA's economy and transport board chairman, said: "The concessionary fares scheme provides a lifeline for our most vulnerable residents to go shopping, pick up medication, attend doctor appointments or socialise with friends. However, it is now under real threat.
"Underfunding of the scheme forces councils to spend millions of pounds of taxpayers' money to subsidise the scheme.
"This is now impossible with councils having to make savings while struggling to protect vital services like adult social care, protecting children, filling potholes and collecting bins."