This article examined the development of an instrument that assessed the automation and usability of clinical information technologies (CIT) in hospitals. Test records or notes being kept electronically indicated automation. Usability referred to perceptions of the effectiveness, ease and user-support aspects of CITs. The measure was pilot-tested with 117 physicians and three chief information officers at four hospitals. Two hospitals were community-based with low investment in CITs while the other hospitals (one academic and one a Veterans Affairs hospital) had more advanced CITs.
- Across all automation domains, physicians' reports were significantly higher in the hospitals that spent more of their resources on CITs.
- Hospitals who invested more in CITs had significantly higher usability scores according to physicians.
- Hospitals that had invested fewer resources in CITs had significantly lower effectiveness subdomain scores per physicians' reports. Scores did not significantly differ on the ease subdomain however.
- Physicians' mean scores on the overall CIT index were significantly higher in hospitals that had more advanced CITs.
- Scores for chief information officers on the overall CIT index and across domains corresponded to the relationships seen with physicians.
- The CIT index was found to be reliable and to have demonstrated construct validity. Factor analysis determined that the domains of the index were distinct.