Marc Pinski
ABSTRACT
When collaborating with artificial intelligence (AI), humans can often delegate tasks to leverage complementary AI competencies. However, humans often delegate inefficiently. Enabling humans with knowledge about AI can potentially improve inefficient AI delegation. We conducted a between-subjects experiment (two groups, n = 111) to examine how enabling humans with AI knowledge can improve AI delegation in human-AI collaboration. We find that AI knowledge-enabled humans align their delegation decisions more closely with their assessment of how suitable a task is for humans or AI (i.e., task appraisal). We show that delegation decisions closely aligned with task appraisal increase task performance. However, we also find that AI knowledge lowers future intentions to use AI, suggesting that AI knowledge is not strictly positive for human-AI collaboration. Our study contributes to HCI design guidelines with a new perspective on AI features, educating humans regarding general AI functioning and their own (human) performance and biases.
Footnotes
1 To reliably detect significant results, we aimed to exceed the minimum sample size of 51 in each experimental group, which was estimated through a power analysis leveraging G*Power 3.1 [33]. Therefore, we used the following parameter: a moderate effect size (f = 0.50), an α-level of 0.05, and a desired power level of 0.80 [26].
Footnote
Supplemental Material
- Martin Adam, Konstantin Roethke and Alexander Benlian. 2022. Human Versus Automated Sales Agents: How and Why Customer Responses Shift Across Sales Stages. Information Systems Research, Forthcoming (2022). https://doi.org/10.1287/isre.2022.1171Google Scholar
Digital Library
- Pär J. Ågerfalk. 2020. Artificial intelligence as digital agency. European Journal of Information Systems, 29, 1 (2020), 1-8. https://doi.org/10.1080/0960085x.2020.1721947Google ScholarCross Ref
- Amazon. 2017. Tutorial: How to label thousands of images using the crowd. Retrieved August 10, 2022 from https://blog.mturk.com/tutorial-how-to-label-thousands-of-images-using-the-crowd-bea164ccbefcGoogle Scholar
- Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul N. Bennett, Kori Inkpen, Jaime Teevan, Ruth Kikin-Gil and Eric Horvitz. 2019. Guidelines for Human-AI Interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3290605.3300233Google ScholarDigital Library
- Eduard Anton, Alina Behne and Frank Teuteberg. 2020. The humans behind artificial intelligence - An operationalisation of AI competencies. In Proceedings of the European Conference on Information Systems. Virtual. https://aisel.aisnet.org/ecis2020_rp/141Google Scholar
- Alejandro Barredo Arrieta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador Garcia, Sergio Gil-Lopez, Daniel Molina, Richard Benjamins, Raja Chatila and Francisco Herrera. 2020. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion, 58 (2020), 82-115. https://doi.org/10.1016/j.inffus.2019.12.012Google ScholarDigital Library
- Sangeeta Bharadwaj Badal and Bryant Ott. 2015. Delegating: A Huge Management Challenge for Entrepreneurs. Retrieved August 10, 2022 from https://news.gallup.com/businessjournal/182414/delegating-huge-management-challenge-entrepreneurs.aspxGoogle Scholar
- Aaron Baird and Likoebe M. Maruping. 2021. The Next Generation of Research on IS Use: A Theoretical Framework of Delegation to and from Agentic IS Artifacts. MIS Quarterly, 45, 1 (2021), 315-341. https://doi.org/10.25300/misq/2021/15882Google ScholarCross Ref
- Gagan Bansal, Besmira Nushi, Ece Kamar, Daniel S. Weld, Walter S. Lasecki and Eric Horvitz. 2019. Update in Human-AI Team. Understanding and Addressing the performance compatibility tradeoff. In Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence (AAAI-19). Honolulu, USA.Google Scholar
- Gagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro and Daniel Weld. 2021. Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3411764.3445717Google ScholarDigital Library
- Reuben M. Baron and David A. Kenny. 1986. The moderator-mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations. J Pers Soc Psychol, 51, 6 (1986), 1173-1182. https://doi.org/10.1037//0022-3514.51.6.1173Google ScholarCross Ref
- Geneviève Bassellier, Izak Benbasat and Blaize Horner Reich. 2003. The Influence of Business Managers' IT Competence on Championing IT. Information Systems Research, 14, 4 (2003), 317-336. https://doi.org/10.1287/isre.14.4.317.24899Google ScholarDigital Library
- Geneviève Bassellier, Blaize Horner Reich and Izak Benbasat. 2015. Information Technology Competence of Business Managers: A Definition and Research Model. Journal of Management Information Systems, 17, 4 (2015), 159-182. https://doi.org/10.1080/07421222.2001.11045660Google ScholarDigital Library
- Kevin Bauer, Moritz von Zahn and Oliver Hinz. Forthcoming. Expl(AI)ned: The Impact of Explainable Articial Intelligence on Users' Information Processing. Information Systems Research (Forthcoming). https://doi.org/10.1287/xxxx.0000.0000Google ScholarCross Ref
- Anne Beaudry and Alain Pinsonneault. 2005. Understanding User Responses to Information Technology: A Coping Model of User Adaptation. MIS Quarterly, 29, 3 (2005). https://doi.org/10.2307/25148693Google ScholarCross Ref
- Hind Benbya, Stella Pachidi and Sirkka L. Jarvenpaa. 2021. Special Issue Editorial: Artificial Intelligence in Organizations: Implications for Information Systems Research. Journal of the Association for Information Systems, 22, 2 (2021), 281-303. https://doi.org/10.17705/1jais.00662Google ScholarCross Ref
- Nicholas Berente, Bin Gu, Jan Recker and Radhika Santhanam. 2021. Managing Artificial Intelligence. MIS Quarterly, 45, 3 (2021), 1433-1450. https://doi.org/10.25300/MISQ/2021/16274Google ScholarCross Ref
- Anol Bhattacherjee. 2001. Understanding information systems continuance: An expectation-confirmation model. MIS Quarterly, 25, 3 (2001), 351-370. https://doi.org/10.2307/3250921Google ScholarDigital Library
- Anol Bhattacherjee, Christopher J. Davis, Amy J. Connolly, Neset Hikmet, Frantz Rowe and Régis Meissonier. 2017. User response to mandatory IT use: a coping theory perspective. European Journal of Information Systems, 27, 4 (2017), 395-414. https://doi.org/10.1057/s41303-017-0047-0Google ScholarCross Ref
- Carrie J. Cai, Emily Reif, Narayan Hegde, Jason Hipp, Been Kim, Daniel Smilkov, Martin Wattenberg, Fernanda Viegas, Greg S. Corrado, Martin C. Stumpe and Michael Terry. 2019. Human-Centered Tools for Coping with Imperfect Algorithms During Medical Decision-Making. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3290605.3300234Google ScholarDigital Library
- Wanling Cai, Yucheng Jin and Li Chen. 2022. Impacts of Personal Characteristics on User Trust in Conversational Recommender Systems. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3491102.3517471Google ScholarDigital Library
- Longbing Cao. 2020. AI in Finance: A Review. SSRN Electronic Journal (2020). https://doi.org/10.2139/ssrn.3647625Google ScholarCross Ref
- Noel Carroll. 2021. Augmented Intelligence: An Actor-Network Theory Perspective. In Proceedings of the European Conference on Information Systems. Marrakesh, Morocco. https://aisel.aisnet.org/ecis2021_rp/37Google Scholar
- Dilek Cetindamar, Kirsty Kitto, Mengjia Wu, Yi Zhang, Babak Abedin and Simon Knight. 2022. Explicating AI Literacy of Employees at Digital Workplaces. IEEE Transactions on Engineering Management (2022), 1-14. https://doi.org/10.1109/tem.2021.3138503Google ScholarCross Ref
- Hyesun Choung, Prabu David and Arun Ross. 2022. Trust in AI and Its Role in the Acceptance of AI Technologies. International Journal of Human–Computer Interaction (2022), 1-13. https://doi.org/10.1080/10447318.2022.2050543Google ScholarCross Ref
- Clifford C. Clogg, Eva Petkova and Adamantios Haritou. 1995. Statistical Methods for Comparing Regression Coefficients Between Models. American Journal of Sociology, 100, 5 (1995), 1261-1293. https://doi.org/10.1086/230638Google ScholarCross Ref
- Jacob Cohen, Patricia Cohen, Stephan G. West and Leona S. Aiken. 2003. Applied multiple regression/correlation analysis for the behavioral sciences. Erlbaum. Hillsdale, NJGoogle Scholar
- W. Alec Cram, Jeffrey G. Proudfoot and John D'Arcy. 2020. When enough is enough: Investigating the antecedents and consequences of information security fatigue. Information Systems Journal, 31, 4 (2020), 521-549. https://doi.org/10.1111/isj.12319Google ScholarCross Ref
- Dominik Dellermann, Philipp Ebel, Matthias Söllner and Jan Marco Leimeister. 2019. Hybrid Intelligence. Business & Information Systems Engineering, 61, 5 (2019), 637-643. https://doi.org/10.1007/s12599-019-00595-2Google ScholarCross Ref
- Xianghua Ding, Yanqi Jiang, Xiankang Qin, Yunan Chen, Wenqiang Zhang and Lizhe Qi. 2019. Reading Face, Reading Health. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3290605.3300435Google ScholarDigital Library
- Mateusz Dolata, Stefan Feuerriegel and Gerhard Schwabe. 2021. A sociotechnical view of algorithmic fairness. Information Systems Journal, 32, 4 (2021), 754-818. https://doi.org/10.1111/isj.12370Google ScholarCross Ref
- Philipp Ebel, Matthias Söllner, Jan Marco Leimeister, Kevin Crowston and Gert-Jan de Vreede. 2021. Hybrid intelligence in business networks. Electronic Markets, 31, 2 (2021), 313-318. https://doi.org/10.1007/s12525-021-00481-4Google ScholarCross Ref
- Christian Engel, Philipp Ebel and Jan Marco Leimeister. 2022. Cognitive automation. Electronic Markets, 32, 1 (2022), 339-350. https://doi.org/10.1007/s12525-021-00519-7Google ScholarCross Ref
- Yoram Eshet-Alkalai. 2004. Digital literacy. Journal of Educational Multimedia and Hypermedia, 13, 1 (2004), 93-106.Google Scholar
- Susan Folkman. 2013. Stress: Appraisal and Coping. In Encyclopedia of Behavioral Medicine. M. D. Gellman and J. R. Turner (Eds.). Springer. New York.Google Scholar
- Claes Fornell and David F. Larcker. 1981. Evaluating Structural Equation Models with Unobservable Variables and Measurement Error. Journal of Marketing Research, 18, 1 (1981). https://doi.org/10.2307/3151312Google ScholarCross Ref
- Andreas Fügener, Jörn Grahl, Alok Gupta and Wolfgang Ketter. 2021. Cognitive Challenges in Human–Artificial Intelligence Collaboration: Investigating the Path Toward Productive Delegation. Information Systems Research (2021). https://doi.org/10.1287/isre.2021.1079Google ScholarDigital Library
- Andreas Fügener, Jörn Grahl, Alok Gupta and Wolfgang Ketter. 2021. Will Humans-in-the-Loop Become Borgs? Merits and Pitfalls of Working with AI. MIS Quarterly, 45, 3 (2021), 1527-1556. https://doi.org/10.25300/misq/2021/16553Google ScholarCross Ref
- Google. 2022. Advanced Guide to Inception v3. Retrieved July 12, 2022 from https://cloud.google.com/tpu/docs/inception-v3-advancedGoogle Scholar
- Sophia Hadash, Martijn C. Willemsen, Chris Snijders and Wijnand A. Ijsselsteijn. 2022. Improving understandability of feature contributions in model-agnostic explainable AI tools. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3491102.3517650Google ScholarDigital Library
- Kotaro Hara, Abigail Adams, Kristy Milland, Saiph Savage, Chris Callison-Burch and Jeffrey P. Bigham. 2018. A Data-Driven Analysis of Workers' Earnings on Amazon Mechanical Turk. In Proceedings of the Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3173574.3174023Google ScholarDigital Library
- Teresa Heyder and Oliver Posegga. 2021. Extending the foundations of AI literacy. In Proceedings of the International Conference on Information Systems. Austin, USA. https://aisel.aisnet.org/icis2021/is_future_work/is_future_work/9Google Scholar
- Bengt Holmström. 1979. Moral Hazard and Observability. The Bell Journal of Economics, 10, 1 (1979). https://doi.org/10.2307/3003320Google ScholarCross Ref
- Kristina Höök. 2000. Steps to take before intelligent user interfaces become real. Interacting with Computers, 12, 4 (2000), 409-426. https://doi.org/10.1016/s0953-5438(99)00006-5Google ScholarCross Ref
- Qian Hu, Yaobin Lu, Zhao Pan, Yeming Gong and Zhilin Yang. 2021. Can AI artifacts influence human cognition? The effects of artificial autonomy in intelligent personal assistants. International Journal of Information Management, 56 (2021). https://doi.org/10.1016/j.ijinfomgt.2020.102250Google ScholarDigital Library
- Jacob Jacoby. 1984. Perspectives on Information Overload. Journal of Consumer Research, 10, 4 (1984). https://doi.org/10.1086/208981Google ScholarCross Ref
- Hemant Jain, Balaji Padmanabhan, Paul A. Pavlou and T. S. Raghu. 2021. Editorial for the Special Section on Humans, Algorithms, and Augmented Intelligence: The Future of Work, Organizations, and Society. Information Systems Research, 32, 3 (2021), 675-687. https://doi.org/10.1287/isre.2021.1046Google ScholarDigital Library
- Ekaterina Jussupow, Kai Spohrer, Armin Heinzl and Joshua Gawlitza. 2021. Augmenting Medical Diagnosis Decisions? An Investigation into Physicians’ Decision-Making Process with Artificial Intelligence. Information Systems Research, 32, 3 (2021), 713-735. https://doi.org/10.1287/isre.2020.0980Google ScholarDigital Library
- Daniel Kahneman and Amos Tversky. 1984. Choices, values, and frames. American Psychologist, 39, 4 (1984), 341-350. https://doi.org/10.1037/0003-066x.39.4.341Google ScholarCross Ref
- Georgi Dimov Kerpedzhiev, Ulrich Matthias König, Maximilian Röglinger and Michael Rosemann. 2020. An Exploration into Future Business Process Management Capabilities in View of Digitalization. Business & Information Systems Engineering, 63, 2 (2020), 83-96. https://doi.org/10.1007/s12599-020-00637-0Google ScholarCross Ref
- Alison King. 1992. Comparison of Self-Questioning, Summarizing, and Notetaking-Review as Strategies for Learning From Lectures. American Educational Research Journal, 29, 2 (1992), 303-323. https://doi.org/10.3102/00028312029002303Google ScholarCross Ref
- Rafal Kocielnik, Saleema Amershi and Paul N. Bennett. 2019. Will You Accept an Imperfect AI? In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3290605.3300641Google ScholarDigital Library
- Vivian Lai, Samuel Carton, Rajat Bhatnagar, Q. Vera Liao, Yunfeng Zhang and Chenhao Tan. 2022. Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3491102.3501999Google ScholarDigital Library
- Richard S. Lazarus and Susan Folkman. 1984. Stress, appraisal and coping. Springer. New YorkGoogle Scholar
- Carrie R. Leana. 1986. Predictors and Consequences of Delegation. Academy of Management Journal, 29, 4 (1986), 754-774. https://doi.org/10.2307/255943Google ScholarCross Ref
- Carrie R. Leana. 1987. Power relinquishment versus power sharing: Theoretical clarification and empirical comparison of delegation and participation. Journal of Applied Psychology, 72, 2 (1987), 228-233. https://doi.org/10.1037/0021-9010.72.2.228Google ScholarCross Ref
- Minsun Lee and Jee‐Sun Park. 2022. Do parasocial relationships and the quality of communication with AI shopping chatbots determine middle‐aged women consumers' continuance usage intentions? Journal of Consumer Behaviour, 21, 4 (2022), 842-854. https://doi.org/10.1002/cb.2043Google ScholarCross Ref
- Claire Liang, Julia Proft, Erik Andersen and Ross A. Knepper. 2019. Implicit Communication of Actionable Information in Human-AI teams. In Proceedings of the Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3290605.3300325Google ScholarDigital Library
- Q. Vera Liao, Daniel Gruen and Sarah Miller. 2020. Questioning the AI: Informing Design Practices for Explainable AI User Experiences. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3313831.3376590Google ScholarDigital Library
- Duri Long and Brian Magerko. 2020. What is AI Literacy? Competencies and Design Considerations. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. Honolulu, USA.Google ScholarDigital Library
- Brian Lubars and Chenhao Tan. 2019. Ask Not What AI Can Do, But What AI Should Do: Towards a Framework of Task Delegability. In Proceedings of the 33rd Conference on Neural Information Processing Systems. Vancouver, Canada.Google Scholar
- Scott B. MacKenzie, Philip M. Podsakoff and Nathan P. Podsakoff. 2011. Construct Measurement and Validation Procedures in MIS and Behavioral Research: Integrating New and Existing Techniques. MIS Quarterly, 35, 2 (2011). https://doi.org/10.2307/23044045Google ScholarCross Ref
- Naresh K. Malhotra. 1982. Information Load and Consumer Decision Making. Journal of Consumer Research, 8, 4 (1982). https://doi.org/10.1086/208882Google ScholarCross Ref
- Connor J. McCabe, Dale S. Kim and Kevin M. King. 2018. Improving Present Practices in the Visual Display of Interactions. Adv Methods Pract Psychol Sci, 1, 2 (2018), 147-165. https://doi.org/10.1177/2515245917746792Google ScholarCross Ref
- Nathan J. McNeese, Beau G. Schelble, Lorenzo Barberis Canonico and Mustafa Demir. 2021. Who/What Is My Teammate? Team Composition Considerations in Human–AI Teaming. IEEE Transactions on Human-Machine Systems, 51, 4 (2021), 288-299. https://doi.org/10.1109/thms.2021.3086018Google ScholarCross Ref
- Patrick Mikalef, Kieran Conboy, Jenny Eriksson Lundström and Aleš Popovič. 2022. Thinking responsibly about responsible AI and ‘the dark side’ of AI. European Journal of Information Systems, 31, 3 (2022), 257-268. https://doi.org/10.1080/0960085x.2022.2026621Google ScholarCross Ref
- Allen E. Milewski and Steven H. Lewis. 1997. Delegating to software agents. International Journal of Human-Computer Studies, 46, 4 (1997), 485-500. https://doi.org/10.1006/ijhc.1996.0100Google ScholarDigital Library
- Franklin G. Moore. 1982. The Management of Organizations. Wiley. New YorkGoogle Scholar
- Alex Murray, Jen Rhymer and David G. Sirmon. 2021. Humans and Technology: Forms of Conjoined Agency in Organizations. Academy of Management Review, 46, 3 (2021), 552-571. https://doi.org/10.5465/amr.2019.0186Google ScholarCross Ref
- Jordan Navarro, Sarah Allali, Nicolas Cabrignac and Julien Cegarra. 2021. Impact of Pilot's Expertise on Selection, Use, Trust, and Acceptance of Automation. IEEE Transactions on Human-Machine Systems, 51, 5 (2021), 432-441. https://doi.org/10.1109/thms.2021.3090199Google ScholarCross Ref
- Franz Neyer, Juliane Felber and Claudia Gebhardt. 2016. Kurzskala zur Erfassung von Technikbereitschaft (technology commitment). Zusammenstellung sozialwissenschaftlicher Items und Skalen (ZIS) (2016). https://doi.org/10.6102/zis244Google ScholarCross Ref
- Davy Tsz Kit Ng, Jac Ka Lok Leung, Samuel Kai Wah Chu and Maggie Shen Qiao. 2021. Conceptualizing AI literacy: An exploratory review. Computers and Education: Artificial Intelligence, 2 (2021). https://doi.org/10.1016/j.caeai.2021.100041Google ScholarCross Ref
- Andy Nguyen, Yvonne Hong and Lesley A. Gardner. 2020. A Taxonomy of Digital Learning Activities for Digital Inclusion. In Proceedings of the European Conference on Information Systems. Virtual. https://aisel.aisnet.org/ecis2020_rp/135Google Scholar
- Tatsuya Nomura, Tomohiro Suzuki, Takayuki Kanda and Kensuke Kato. 2006. Measurement of negative attitudes toward robots. Interaction Studies, 7, 3 (2006), 437-454. https://doi.org/10.1075/is.7.3.14nomGoogle ScholarCross Ref
- Ikujiro Nonaka. 1994. A dynamic theory of organizational knowledge creation. Organization Science, 5, 1 (1994), 14-37. http://www.jstor.com/stable/2635068Google ScholarDigital Library
- Milda Norkute, Nadja Herger, Leszek Michalak, Andrew Mulder and Sally Gao. 2021. Towards Explainable AI: Assessing the Usefulness and Impact of Added Explainability Features in Legal Document Summarization. In Proceedings of the Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3411763.3443441Google ScholarDigital Library
- Philip M. Podsakoff, Scott B. MacKenzie and Nathan P. Podsakoff. 2016. Recommendations for Creating Better Concept Definitions in the Organizational, Behavioral, and Social Sciences. Organizational Research Methods, 19, 2 (2016), 159-203. https://doi.org/10.1177/1094428115624965Google ScholarCross Ref
- Qing Rao and Jelena Frtunikj. 2018. Deep learning for self-driving cars. In Proceedings of the 1st International Workshop on Software Engineering for AI in Autonomous Systems. Gothenburg, Sweden.Google ScholarDigital Library
- Sebastian Schuetz and Viswanath Venkatesh. 2020. Research Perspectives: The Rise of Human Machines: How Cognitive Computing Systems Challenge Assumptions of User-System Interaction. Journal of the Association for Information Systems (2020), 460-482. https://doi.org/10.17705/1jais.00608Google ScholarCross Ref
- Axel Schulte, Diana Donath and Douglas S. Lange. 2016. Design Patterns for Human-Cognitive Agent Teaming. In Engineering Psychology and Cognitive Ergonomics. D. Harris (Ed.). Springer. Cham.Google Scholar
- Haifeng Shen, Kewen Liao, Zhibin Liao, Job Doornberg, Maoying Qiao, Anton van den Hengel and Johan W. Verjans. 2021. Human-AI Interactive and Continuous Sensemaking: A Case Study of Image Classification using Scribble Attention Maps. In Proceedings of the Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3411763.3451798Google ScholarDigital Library
- Herbert A. Simon. 1990. Bounded rationality. In Utility and Probability. J. Eatwell, M. Milgate and P. Newman (Eds.). Palgrave Macmillan. London.Google Scholar
- Ida Someh, Michael Davern, Christoph F. Breidbach and Graeme Shanks. 2019. Ethical Issues in Big Data Analytics: A Stakeholder Perspective. Communications of the Association for Information Systems (2019), 718-747. https://doi.org/10.17705/1cais.04434Google ScholarCross Ref
- Stanford Vision Labs. 2022. ImageNet. Retrieved July 12, 2022 from https://www.image-net.org/index.phpGoogle Scholar
- Frederik Stjernfelt and Anne Mette Lauritzen. 2020. Facebook's Handbook of Content Removal. In Your Post Has Been Removed. Springer. Cham.Google Scholar
- Yi Sun, Shihui Li and Lingling Yu. 2021. The dark sides of AI personal assistant: effects of service failure on user continuance intention. Electronic Markets, 32, 1 (2021), 17-39. https://doi.org/10.1007/s12525-021-00483-2Google ScholarCross Ref
- Thomas Süße, Maria Kobert and Caroline Kries. 2021. Antecedents of Constructive Human-AI Collaboration: An Exploration of Human Actors’ Key Competencies. In Smart and Sustainable Collaborative Networks 4.0. p. 113-124.Google Scholar
- Monideepa Tarafdar, Xinru Page and Marco Marabelli. 2022. Algorithms as co‐workers: Human algorithm role interactions in algorithmic work. Information Systems Journal (2022). https://doi.org/10.1111/isj.12389Google ScholarCross Ref
- Mike Teodorescu, Lily Morse, Yazeed Awwad and Gerald Kane. 2021. Failures of Fairness in Automation Require a Deeper Understanding of Human-ML Augmentation. MIS Quarterly, 45, 3 (2021), 1483-1500. https://doi.org/10.25300/misq/2021/16535Google ScholarCross Ref
- Scott Thiebes, Sebastian Lins and Ali Sunyaev. 2020. Trustworthy artificial intelligence. Electronic Markets, 31, 2 (2020), 447-464. https://doi.org/10.1007/s12525-020-00441-4Google ScholarCross Ref
- James Y. L. Thong, Se-Joon Hong and Kar Yan Tam. 2006. The effects of post-adoption beliefs on the expectation-confirmation model for information technology continuance. International Journal of Human-Computer Studies, 64, 9 (2006), 799-810. https://doi.org/10.1016/j.ijhcs.2006.05.001Google ScholarDigital Library
- Güliz Tokadlı, Michael C. Dorneich and Michael Matessa. 2021. Evaluation of Playbook Delegation Approach in Human-Autonomy Teaming for Single Pilot Operations. International Journal of Human–Computer Interaction, 37, 7 (2021), 703-716. https://doi.org/10.1080/10447318.2021.1890485Google ScholarCross Ref
- Eric J. Topol. 2019. High-performance medicine: the convergence of human and artificial intelligence. Nat Med, 25, 1 (Jan 2019), 44-56. https://doi.org/10.1038/s41591-018-0300-7Google ScholarCross Ref
- P. Tschandl, C. Rinner, Z. Apalla, G. Argenziano, N. Codella, A. Halpern, M. Janda, A. Lallas, C. Longo, J. Malvehy, J. Paoli, S. Puig, C. Rosendahl, H. P. Soyer, I. Zalaudek and H. Kittler. 2020. Human-computer collaboration for skin cancer recognition. Nat Med, 26, 8 (2020), 1229-1234. https://doi.org/10.1038/s41591-020-0942-0Google ScholarCross Ref
- Amos Tversky and Daniel Kahneman. 1974. Judgment under Uncertainty: Heuristics and Biases. Science, 185, 4157 (1974), 1124–1131. http://www.jstor.org/stable/1738360Google Scholar
- Benjamin van Giffen, Dennis Herhausen and Tobias Fahse. 2022. Overcoming the pitfalls and perils of algorithms: A classification of machine learning biases and mitigation methods. Journal of Business Research, 144 (2022), 93-106. https://doi.org/10.1016/j.jbusres.2022.01.076Google ScholarCross Ref
- Jhim Kiel M. Verame, Enrico Costanza, Joel Fischer, Andy Crabtree, Sarvapali D. Ramchurn, Tom Rodden and Nicholas R. Jennings. 2018. Learning from the Veg Box. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3173574.3174021Google ScholarDigital Library
- Michael Vössing, Niklas Kühl, Matteo Lind and Gerhard Satzger. 2022. Designing Transparency for Effective Human-AI Collaboration. Information Systems Frontiers, 24, 3 (2022), 877-895. https://doi.org/10.1007/s10796-022-10284-3Google ScholarDigital Library
- Bingcheng Wang, Pei-Luen Patrick Rau and Tianyi Yuan. 2022. Measuring user competence in using artificial intelligence: validity and reliability of artificial intelligence literacy scale. Behaviour & Information Technology (2022), 1-14. https://doi.org/10.1080/0144929x.2022.2072768Google ScholarCross Ref
- Danding Wang, Qian Yang, Ashraf Abdul and Brian Y. Lim. 2019. Designing Theory-Driven User-Centric Explainable AI. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3290605.3300831Google ScholarDigital Library
- Leslie Willcocks. 2020. Robo-Apocalypse cancelled? Reframing the automation and future of work debate. Journal of Information Technology, 35, 4 (2020), 286-302. https://doi.org/10.1177/0268396220925830Google ScholarCross Ref
- Michael Williams, Hassen Gharbi, Aybike Ulusan, Ozlem Ergun, Zhu Xiaofeng, Shiyu Zhang and Casper Harteveld. 2016. Toward Human in the Loop Optimization Through Game-Based Experiments. In Proceedings of the Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts. https://doi.org/10.1145/2968120.2987733Google ScholarDigital Library
- World Economic Forum. 2022. Without universal AI literacy, AI will fail us. Retrieved August 10, 2022 from https://www.weforum.org/agenda/2022/03/without-universal-ai-literacy-ai-will-fail-us/Google Scholar
- Y. Xie, I.P. Bodala, D. C. Ong, D. Hsu and H. Soh. 2019. Robot Capability and Intention in Trust-Based Decisions Across Tasks. In Proceedings of the 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI).Google Scholar
- Su-Fang Yeh, Meng-Hsin Wu, Tze-Yu Chen, Yen-Chun Lin, XiJing Chang, You-Hsuan Chiang and Yung-Ju Chang. 2022. How to Guide Task-oriented Chatbot Users, and When: A Mixed-methods Study of Combinations of Chatbot Guidance Types and Timings. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3491102.3501941Google ScholarDigital Library
- Sangseok You, Cathy Liu Yang and Xitong Li. 2022. Algorithmic versus Human Advice: Does Presenting Prediction Performance Matter for Algorithm Appreciation? Journal of Management Information Systems, 39, 2 (2022), 336-365. https://doi.org/10.1080/07421222.2022.2063553Google ScholarCross Ref
- Gary Yukl and Ping P. Fu. 1999. Determinants of Delegation and Consultation by Managers. Journal of Organizational Behavior, 20, 2 (1999), 219-232. https://www.jstor.org/stable/3100422Google ScholarCross Ref
- Jichen Zhu, Jennifer Villareale, Nithesh Javvaji, Sebastian Risi, Mathias Löwe, Rush Weigelt and Casper Harteveld. 2021. Player-AI Interaction: What Neural Network Games Reveal About AI as Play. In Proceedings of the Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3411764.3445307Google ScholarDigital Library
Index Terms
- AI Knowledge: Improving AI Delegation through Human Enablement
Recommendations
Human-AI Collaboration: The Effect of AI Delegation on Human Task Performance and Task Satisfaction
IUI '23: Proceedings of the 28th International Conference on Intelligent User InterfacesRecent work has proposed artificial intelligence (AI) models that can learn to decide whether to make a prediction for an instance of a task or to delegate it to a human by considering both parties’ capabilities. In simulations with synthetically ...
AI Literacy: Finding Common Threads between Education, Design, Policy, and Explainability
CHI EA '23: Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing SystemsFostering public AI literacy has been a growing area of interest at CHI for several years, and a substantial community is forming around issues such as teaching children how to build and program AI systems, designing learning experiences to broaden ...
What is AI Literacy? Competencies and Design Considerations
CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing SystemsArtificial intelligence (AI) is becoming increasingly integrated in user-facing technology, but public understanding of these technologies is often limited. There is a need for additional HCI research investigating a) what competencies users need in ...
Comments