The answer is officially, from 1st July until around mid August.
Although of course, you will see people who is swimming not only in this period.
For example, this weekend(31st May 2014), I saw some people in the water but they only put their legs. The temperature was around 34degree.
(Picture from 31st May 2014 at Amanohashidate)