I wonder if it is your DNS configuration? Is the hostname for IPv6 resolving to the correct address?
Try (replace google.com with your hostname):
Code:
tyler@desktop:~/Downloads$ nslookup
> set type=AAAA
> google.com
Server: 8.8.4.4
Address: 8.8.4.4#53
Non-authoritative answer:
google.com has AAAA address 2607:f8b0:4002:c08::71
and verifying your name servers are giving the correct IPv6 address. Also check your virtual host configurations to verify you don't have multiple virtual hosts, one for IPv4 and another for IPv6.