African migrants at sea. Image: Marina Militare via ABC
A top UN official described the shipwreck as "the worst Mediterranean tragedy" so far this year.
Earlier this month, 82 migrants died after their boat capsized off the coast of Tunisia.
About 164 people died on the perilous route between Libya and Europe in the first four months of 2019, UNHCR figures show.
The refugee agency said the sunken boat had taken to the sea from Libya, where renewed warfare between rival factions has gripped the capital Tripoli in the past three months.
The boat was carrying some 300 people from several African and Arab countries when it sank 8 kilometres from the coast, around 120km east of Tripoli.
"We estimate that 150 migrants are potentially missing and died at sea," UNHCR spokesman Charlie Yaxley said, with 147 rescued.
"The dead include women and children."
Mr Yaxley added that so far this year, one person has died on the route from Libya to Europe for every six people that reached Europe shores.
He called on European nations to resume rescue missions in the Mediterranean, which were halted after an EU decision, and appealed for an end to migrant detentions in Libya.
Mr Grandi said safe pathways out of the North African country are needed "before it is too late for many more desperate people."
A mass exodus from northern Africa to Europe began after the 2011 uprisings that toppled and killed the longtime Libyan dictator Muammar Gaddafi.
Traffickers and armed groups have exploited Libya's chaos since his overthrow, and have been implicated in widespread abuses of migrants, including torture and abduction for ransom.
The migrant route has claimed thousands of lives, but there has been a significant reduction in refugees and migrants risking the crossing since Italy, backed by the EU, spent millions on strengthening the Libyan coast guard to intercept the boats.
Earlier this week, the Libyan coast guard intercepted around three dozen migrants off the coast and took them to a detention centre near Tripoli where an airstrike killed at least 44 people.
Over 200 detainees are still being held at the Tajoura detention centre, near the front lines of fighting between rival Libyan factions. The UN has voiced concern for their safety.
The International Rescue Committee said the tragedy was a stark reminder of the humanitarian crisis emerging out of Libya and of the urgent need for search and rescue missions to be resumed in the Mediterranean.
459 people are talking about this