CDPJ leader Yoshihiko Noda has said he would work with other parties to try and oust the incumbents, though analysts see this as a more remote possibility.
The LDP has ruled Japan for almost all of its post-war history and the result marked its worst election since it briefly lost power in 2009 to a precursor of the CDPJ.
Ishiba, picked in a close-fought race to lead the LDP late last month, called the election a year before it was due in an effort to secure a public mandate.
His initial ratings suggested he may be able to capitalise on his personal popularity, but like his predecessor Fumio Kishida he was undone by resentment over his handling of a scandal involving unrecorded donations to LDP lawmakers.
Ishiba's LDP declined to endorse several scandal-tainted candidates in the election. But days before the vote, a newspaper affiliated with the Japan Communist Party reported that the LDP had provided campaign funds to branches headed by non-endorsed candidates.
The story was picked up widely by Japanese media despite Ishiba saying the money could not be used by non-endorsed candidates. "LDP's payments to branches show utter lack of care for public image," ran an editorial in the influential Asahi newspaper two days before the election.
Support from smaller parties, such as the Democratic Party for the People (DPP) or the Japan Innovation Party (JIP), which won 28 and 38 seats respectively, could now be key for the LDP.
DPP chief Yuichiro Tamaki and JIP leader Nobuyuki Baba have both said they would rule out joining the coalition but are open to ad hoc cooperation on certain issues.
Ishiba echoed that sentiment, saying "at this moment in time, we are not anticipating a coalition" with other opposition parties. The LDP would hold discussions with other parties and possibly take on some of their policy ideas, he added.
The DPP and JIP propose policies that could be challenging for the LDP and the Bank of Japan.
The DPP calls for halving Japan's 10% sales tax until real wages rise, a policy not endorsed by the LDP, while both parties have criticised the BOJ's efforts to raise interest rates and wean Japan off decades of monetary stimulus.
"It's up to what can they give to these two parties to try and get them to just kind of join their side. The best scenario is getting them into the coalition government, but that's a tall order," said Rintaro Nishimura, an associate at consultancy The Asia Group.
In a statement, the head of Japan's most powerful business lobby Keidanren, Masakazu Tokura, said he hoped for a stable government centred on the LDP-Komeito coalition to steer an economy that faces urgent tasks such as boosting energy security and maintaining the momentum for wage hikes.
In one bright spot, a record 73 women were elected into Japan's male-dominated parliament, surpassing 54 at the 2009 election.
Source: Reuters