White Birch dropping leaves

[Cord-n-8R]

Hi, I have a white birch clump (3 stems) which is probably 35-40 yrs old. I'm in so. Wisconsin. Just in last few days I've been noticing it's been dropping quit a few leaves. The leaves are not yellow or brown and don't appear to be chewed off (or on :D). I've also noticed a few leaves on the tips of branches that have curled and turned brown. 
I have also had a Japanese Beetle problem the last couple years but haven't seen them yet this year.
Can anyone tell me why these leaves might be dropping?
Thanks in advance.

[Southside]

Did you get a late frost?  We did and it had an impact.  

[Cord-n-8R]

We did have late frost but I can't recall if it was before or after the buds started opening.

Edit: just looked it up... Frost was probably 2 1/2 - 3 weeks ago... so, yeah probably about the time the buds/leaves were starting to open.

[Bricklayer51]

Got [a [maple doing the same after the winds last nite the lawn is covered with leaves

[ppine]

Is it a native birch or European white birch.  Birch borers commonly take out these trees before maturity.  Look for holes in the bark, and small piles of sawdust at the base.