It appears the Japanese government does not want to do President Moon any favors before he leaves office which will likely influence whether he visits the Olympics or not:
President Moon Jae-in has a hard decision to make on whether to visit Japan on the occasion of the Tokyo Olympics as the envisaged trip is drawing a mixed response.
Korea Times
Advocates for the President’s participation in the opening ceremony of the quadrennial sporting event, which kicks off July 23, insist that he needs to go in order to break a deadlock in strained bilateral ties, but those critical of this say he should not travel to Tokyo as Japan is taking a lukewarm attitude toward a summit and not showing much resolve in addressing various disputes between the two countries.
Moon is making a last-ditch effort to normalize Korea’s relations with Japan before his term ends in May 2022, as this could enhance trilateral cooperation with the United States. In that sense, the two neighboring countries have been in talks over Moon’s attendance at the opening ceremony, which would lead to his first summit with Japanese Prime Minister Yoshihide Suga and a discussion about pending bilateral issues, according to the foreign ministry here.
However, Tokyo is seemingly unwilling to commit to a meeting, repeatedly leaking information on the negotiations to the media that deprecates a possible summit. Some media reported that any talks between Moon and Suga may last only 15 minutes due to time constraints.
You can read more at the link.