Semiclassical quantization of M5 brane probes wrapped on AdS3× S3 and defect anomalies

We consider two supersymmetric M5 brane probe solutions in AdS7 × S4 and one in AdS4 × S7 that all have the AdS3 × S3 world-volume geometry. The values of the classical action of the first two M5 probes (with S3 in AdS7 or in S4) are related to the leading N2 parts in the anomaly b-coefficient in the (2,0) theory corresponding to a spherical surface defect in symmetric or antisymmetric SU(N) representations. We present a detailed computation of the corresponding one-loop M5 brane partition functions finding that they vanish (in a particular regularization). This implies the vanishing of the order N0 part in the b-anomaly coefficients, in agreement with earlier predictions for their exact values. It remains, however, a puzzle of how to reproduce the non-vanishing order N terms in these coefficients within the semiclassical M5-brane probe setup.