Does Agar Agar expand in the stomach if not diluted enough?

Assuming that you are worried about the expansion rate in the stomach from having an insufficient ratio of water to agar during digestion, a quick search came up with the following results from wikipedia (https://en.wikipedia.org/wiki/Agar):

Agar-agar is approximately 80% fiber, so it can serve as an intestinal regulator. Its bulking quality has been behind fad diets in Asia, for example the kanten (the Japanese word for agar-agar[3]) diet. Once ingested, kanten triples in size and absorbs water. This results in the consumers feeling fuller. This diet has recently received some press coverage in the United States as well. The diet has shown promise in obesity studies.

The same Agar used in culinary dishes is the same one used in laboratories for testing. Resistance to high temperature (melting point is 85 °C) makes Agar ideal for studies that need an environment to mimic human body temperature. This means that Agar will not melt in the body once digested.

Melting Agar is also not the same as denaturing Agar. The act of melting it to form and then cooling to set does not change the structure and properties of Agar. Just like how ice melts into water and you can freeze again while also being able to reverse the process at any time, you can do the same with Agar just at a different temperature. Since the Agar has the same properties as before melting, this does not keep it from expanding when exposed to water.

To add to this: Just because your body does not get hot enough to melt agar, that does not mean the bacteria in your intestine won't break it down into smaller molecules and denature the Agar to aid in digestion. Source: https://www.ayurtimes.com/agar-agar-kanten/

The short answer is: Yes, it can. But it's not really about dilution.

Agar agar, along with foods such as chia seeds and flax seeds, are mucilaginous. This property can indeed pose a hazard if the ingredient is consumed while it still possesses a substantial capacity to absorb water. In this case, the product absorbs moisture from the alimentary canal, which can lead to an obstruction of the gut. There is at least one well-documented case of this occurring with chia seeds.

This risk can be completely mitigated by near-total hydration of the ingredient, i.e., by assuring that its capacity to absorb water has been nearly exhausted. To do so, find the absorptive capacity of the ingredient. My research found that agar agar's capacity is to absorb about 20 times its weight in water. Thus if your finished product contains 1Kg (1 liter) of water and 50 grams of agar agar, you have nothing to worry about. (You should do your own research and not rely on mine.)

I would guess that your finished product is actually substantially dryer than that, and as such, consuming substantial quantities of the product along with dehydration could lead to an obstruction. I would suggest eating this "food" with water.

Like other gelling agents, agar becomes more liquified and hence softer in the presence of the warmth of one's body and the liquid present in the digestive system. Therefor the cooked agar is not in itself a choking hazard, though, as noted above, the gummies could be. It is only agar that has not been combined with liquid, heated, and gelled that it is a choking hazard. So your cooked agar is not a danger. (Though I did notice a distinct flavor from the agar when I made a similar recipe several years ago.)