The biggest concern is the cost incurred for test automation. Also, often s/w test automation requires skilled personnel and authentic test automation tools. Purchasing license of such tools is a costly affair.
Also the time required for test automation is more. Automation often requires recording the sequence of operations you need to perform in order to execute a particular test case and comparing the final o/p with the expected o/p. Functions/macros need also be written sometimes for some test cases. All this could be a time consuming job.