Mr Hodgson decided to stick with Cllr Attenborough as his adult services portfolio holder - a position that she held under previous mayor Frank Branston - but the Tories rejected the offer of children's services despite Cllr Jane Walker holding that position on the unitary council and the previous Bedfordshire County Council.
In a statement on Friday Cllr Ellis said: "We do not believe in dodging difficult decisions and we have taken our first one, which is not to join the Lib Dem Cabinet.
"Cutting council tax is a key public priority and we have sat for years at council meetings and watched as the Lib Dems tried to scupper budget after budget, trying to make policy on the hoof and putting forward financial proposals that could have been very hard to justify to residents.
"That is no way to run a council.
"We know the Lib Dems are a high tax party and we do not believe we can achieve our own priorities to cut council tax by working with a party who want to see a whole new raft of taxes introduced to hit local residents.
"The Conservative vision for the year ahead is very simple. We will leave no stone unturned to cut council tax and deliver what Conservative councils are renowned for value for money and excellent services."
But the decision has been questioned by Mr Hodgson who believes that it was dictated from higher in the party to seek political gain over the best interests of residents.
In a speech following the mayoral by-election, Conservative candidate Parvez Akhtar pledged that the group would support the new mayor as they 'owed it to Frank'.
After telling Mr Hodgson that they would serve on the cabinet they then changed their mind and didn't offer any explanation or attempt of negotiation.
Mr Hodgson said: "It is deeply regrettable that the Conservatives have broken their pledge to work together with me in the interests of Bedford borough residents.
"I have been offered no explanation as to why they have suddenly performed a U-turn and flatly rejected the chance to work together with me and those of all parties and none.
"However, it appears to be an open secret that this decision was dictated to local Conservatives from Tory Party HQ in London.
"They appear to believe that it will benefit their general election chances if they oppose the activities of the council at every turn as we seek to work together to boost the local economy and support residents during these difficult economic times.
"I am absolutely determined to ensure that this cynical political move will not succeed preventing us from achieving the very best for local residents." The Children's Services portfolio has now been filled by Lib Dem councillor David Sawyer.
