SEED: Confidential Big Data Workflow Scheduling with Intel SGX Under Deadline Constraints

Recently, cloud platforms play an essential role in large-scale big data analytics and especially running scientific workflows. In contrast to traditional on-premise computing environments, where the number of resources is bounded, cloud computing can provide practically unlimited resources to a workflow application based on a pay-as-you-go pricing model. One challenge of using cloud computing is the protection of the privacy of the confidential workflow’s tasks, whose proprietary algorithm implementations are intellectual properties of the respective stakeholders. Another one is the monetary cost optimization of executing workflows in the cloud while satisfying a user-defined deadline. In this paper, we use the Intel Software Guard eXtensions (SGX) as a Trusted Execution Environment (TEE) to support the confidentiality of individual workflow tasks. Based on this, we propose a deadline-constrained and SGX-aware workflow scheduling algorithm, called SEED (SGX, Efficient, Effective, Deadline Constrained), to address these two challenges. SEED features several heuristics, including exploiting the longest critical paths and reuse of extra times in existing virtual machine instances. Our experiments show that SEED outperforms the representative algorithm, IC-PCP, in most cases in monetary cost while satisfying the given user-defined deadline. To our best knowledge, this is the first workflow scheduling algorithm that considers protecting the confidentiality of workflow tasks in a public cloud computing environment.