With Pakistan's Khan taking office in August, Pompeo said on the way to the nuclear-armed nation that he wanted "to get out there at the beginning of his time in an effort to reset the relationship between the two countries."
On the plane from the United States, Pompeo told reporters, "There are a lot of challenges between our two nations for sure but we're hopeful that with the new leadership that we can find common ground and we can begin to work on some of our shared problems together."
In Islamabad, Pompeo also met Pakistani Foreign Minister Shah Mehmood Qureshi and is expected to meet the army chief, Qamar Javed Bajwa.
The meetings come against a backdrop of tense ties and U.S. military aid cuts over Islamabad's alleged reluctance to crack down on militants sheltering on its territory.
"Foreign Minister Qureshi underscores the need to reset bilateral ties on the basis of mutual trust and respect," Pakistan's Foreign Ministry spokesman Mohammad Faisal said on Twitter, posting a picture of the meeting between Pompeo and Qureshi.
Faisal said talks on "bilateral, regional and international issues" were held.
Ties between uneasy allies United States and Pakistan have frayed in recent years, with Washington accusing Islamabad of turning a blind eye to, or helping, Afghan Taliban and Haqqani network militants who stage attacks in Afghanistan. Pakistan denies doing so.
Pompeo landed in Islamabad around 1 p.m. (0800 GMT), minutes after the plane carrying U.S. Marine General Joseph Dunford, chairman of the Joint Chiefs of Staff.
Ahead of the talks, Dunford said U.S. President Donald Trump's South Asia strategy set clear expectations for Pakistan, including help to drive the Taliban to a peace process in neighboring Afghanistan.
"Our bilateral relationship moving forward is very much going to be informed by the degree of cooperation we see from Pakistan in doing that," Dunford told reporters.
Taking a tougher line with an ally that Trump considers unreliable, the United States halted the military aid disbursement of the Coalition Support Funds, over Islamabad's perceived failure to rein in Afghan Taliban militants operating from Pakistani soil.
The United States has withheld $800 million in overall assistance this year, cuts Pakistan says are unwarranted as it incurs expenses in fighting militants who pose a danger to U.S. troops in Afghanistan.
Before Wednesday's talks Qureshi said he would "have exchanges" with Pompeo over Washington's cancellation of a $300-million disbursement for the Pakistani military, first reported by Reuters last week.
Pompeo is also expected to discuss Pakistan's possible plans to seek a bailout from the International Monetary Fund (IMF) to ease currency pressures and avert an economic crisis.
In July, Pompeo said there was "no rationale" for the IMF to give money to Pakistan that would then be used to pay off Chinese loans, comments that further rattled Islamabad.
Pompeo and U.S. Defense Secretary Jim Mattis will meet their Indian counterparts in the capital, New Delhi, on Thursday, and are expected to finalise defence pacts that could bring their militaries closer amid China's rising influence.
The talks come as U.S. hostility rises towards India's traditional allies Iran and Russia, on which Washington has placed sanctions. Iran is a big oil supplier to India, and two-thirds of its military equipment is from Russia.
"They're part of the conversation, they're part of the relationship, they will certainly come up, but I don't think they will be the primary focus of what it is we're trying to accomplish here," Pompeo told U.S. pool reporters en route to Pakistan.
The United States is concerned about India's planned purchase of S-400 surface-to-air missile systems from Russia.
An Indian defence ministry official said the country had nearly concluded commercial negotiations with Russia for the systems and intends to proceed with them, to boost defences against China.
On Iran, India has said it will not completely halt oil imports from the Middle Eastern country, but would finalise its strategy on crude purchases after this week's meeting.
(Additional reporting by Drazen Jorgic in Islamabad and Krishna Das in India; Editing by Clarence Fernandez) ((email@example.com; +92 307 8888 153; Reuters Messaging: firstname.lastname@example.org))