U.S. Treasury yields fell to two-week lows on Tuesday after President Donald Trump refrained from imposing tariffs on his first day in office, but said he was thinking about implementing them, keeping investors on edge about inflation.

In his inauguration speech, Trump declared immigration and energy emergencies, but only briefly mentioned tariffs and issued a memo that just directed agencies to investigate and remedy persistent trade deficits.

That stoked expectations the incoming administration will adopt a gradual approach to tariffs, sparking a short-lived relief rally in most non-dollar currencies, with stock futures also soaring before fresh comments from Trump jolted the markets.

Trump said he was thinking about imposing 25% tariffs on imports from Canada and Mexico as soon as Feb. 1, without offering details. Trump also said he wanted to reverse the U.S. trade deficit with the European Union, either with tariffs or more energy exports.

Market ructions in the wake of those comments were mainly felt in currencies, with Treasury yields staying lower as investors awaited further details.

"Our view remains that tariffs are a negotiating tool, and we should see more headlines over the coming days," said Mohit Kumar at Jefferies.

"Eventually, it is likely that Trump refrains from imposing the intended 25% tariffs."

The yield on the benchmark U.S. 10-year Treasury note fell 4 basis points (bps) to 4.5723%, after touching a more than two-week low of 4.53%. The yield on the 30-year bond fell 4 bps to 4.8019%.

The two-year U.S. Treasury yield, which typically moves in step with interest rate expectations, fell less than one bp to 4.2635%. Earlier on Tuesday the yield touched its lowest since Jan. 2 at 4.2190%.

Analysts cautioned that even a measured approach on tariffs could stoke inflation worries and keep U.S. rates higher for longer.

"If you look at what Trump said in his speech, it looks like he's quite firm on tariffs," said Zachary Griffiths, senior investment grade strategist at CreditSights.

"If you have a more gradual, but still large tariffs in terms of percentage on a broad swath of countries ... that could be more challenging from an inflation perspective for the Fed and could even result in policy being tighter for longer," Griffiths said.

The Federal Reserve last month shocked the market by projecting just two rate cuts in 2025, down from four predicted previously, due to worries over inflation and uncertainties surrounding the Trump administration's election pledges.

Analysts have said that Trump's policies on immigration, tax and tariffs will likely boost growth but also be inflationary.

The Fed is expected to hold rates steady this month but keep a wary eye on inflation. Markets price in about 40 basis points of easing this year.

(Reporting by Ankur Banerjee and Greta Rosen Fondahn; Editing by Jamie Freed, Christopher Cushing, Shri Navaratnam and Christina Fincher)