Q. Can we use AuthentiX as well as maintain IIS directory and NTFS security?
A. You could choose to map requests to an NT user for a directory (eg MemProxyUser), see http://www.flicks.com/authentix/discover/access/protectBy.htm
Then remove permissions for that directory for all but that NT user. If someone accidently removed the AuthentiX filter, NTFS would take over.
Also see here.
Also try turning on security auditing for authx.adb, make sure that only the right people are accessing it.
Are you using the internal database? If so make sure you have less than about 5000 users, see here