Let $\bar{a}$ and $\bar{c}$ be unit vectors at an angle $\frac{\pi}{3}$ with each other. If $(\bar{a} \times(\bar{b} \times \bar{c})) \cdot(\bar{a} \times \bar{c})=5$, then $\left[\begin{array}{lll}\bar{a} & \bar{b} & \bar{c}\end{array}\right]=$

The given equation is $(\bar{a} \times(\bar{b} \times \bar{c})) \cdot(\bar{a} \times \bar{c}) = 5$.

Using the BAC-CAB identity, $\bar{a} \times(\bar{b} \times \bar{c}) = (\bar{a} \cdot \bar{c})\bar{b} - (\bar{a} \cdot \bar{b})\bar{c}$.

Substituting into the original equation yields $[(\bar{a} \cdot \bar{c})\bar{b} - (\bar{a} \cdot \bar{b})\bar{c}] \cdot (\bar{a} \times \bar{c}) = 5$.

Expanding the dot product gives $(\bar{a} \cdot \bar{c})[\bar{b} \cdot (\bar{a} \times \bar{c})] - (\bar{a} \cdot \bar{b})[\bar{c} \cdot (\bar{a} \times \bar{c})] = 5$.

Since $\bar{c} \cdot (\bar{a} \times \bar{c}) = 0$ (as two identical vectors in a scalar triple product yield zero), the second term vanishes, simplifying to $(\bar{a} \cdot \bar{c})[\bar{b} \cdot (\bar{a} \times \bar{c})] = 5$.

Given $\bar{a}$ and $\bar{c}$ are unit vectors with an angle of $\frac{\pi}{3}$, $\bar{a} \cdot \bar{c} = 1 \cdot 1 \cdot \cos\frac{\pi}{3} = \frac{1}{2}$.

Substituting, $\frac{1}{2}[\bar{b} \cdot (\bar{a} \times \bar{c})] = 5$, so $\bar{b} \cdot (\bar{a} \times \bar{c}) = 10$.

This expression equals the scalar triple product $[\bar{b} \bar{a} \bar{c}]$, which is related to $[\bar{a} \bar{b} \bar{c}]$ by a sign change: $[\bar{b} \bar{a} \bar{c}] = -[\bar{a} \bar{b} \bar{c}]$.

Thus, $-[\bar{a} \bar{b} \bar{c}] = 10$, giving $[\bar{a} \bar{b} \bar{c}] = -10$.

The correct option is B, and the final answer is $\boxed{-10}$.