File retrieve

Data Flow

As introduced in the Health data sharing and Clinical document sharing technical implementation guidance, we offer two distinct options to facilitate seamless data retrieval from the S3 bucket:

• Option 1: Using a Secure Token Service (STS) Authorization token
• Option 2: Using pre-signed URLs

Option 1: Using a Secure Token Service (STS) Authorization token

Once EMR middleware receives the file availability notification (Step 2), file retrieve service (Step 4) can be called passing as parameter:

• Authorization token;
• MessageReference from Step 2;

The file available for EMR consumption is available in Amazon S3 bucket. Please, follow the steps to use amazon SDK to retrieve the file from amazon S3 bucket:

1. Notification message is received (Step n2);

   Sample Notification message:

   {
       "EmrOid": "emrOID123",
       "MessageId": "f7dc184e-996f-4f8b-934d-52243685b375",
       "MessageType": "Clinical Data",
       "MessageCreatedAt": "1603434071811",
       "MessageReference": {
           "accesspoint": "arn:aws:s3:us-east-1:019364481091:accesspoint/emrOID123",
           "objectKey": "delivery/clinical-data/Sample_insulin.json"
       },
       "MessageReceiptHandle": "AQEB / HM0FfuUJEm3ZW + ss9T9IUo6ONzL / 7261n ZZqmBBpNxemmJwtUCe4IisQdyK2ZaHjhcKFJpLJbh4ge7oWYWA6AOcYlt / +EZSvLH + 6 z9XvQtBQt / jL72jku4NUgixs2uk + 9 UeA1EG + zZs9RFYUHstPIv17yFyI038BbZaKJpZGV + auiWuwd7zrfudWF3cJuCZsiSruFALVtt + RME + VPjF1JOmrfGPZCY3eK2MioAOwjt9yacLezqhTLKl6aaVpzUg1T05hu9NJbnM43 + 4 WdlCm5X7FuAud4knS6BKkFiLvvSEbgxXATjDAPsCn9Ze + Q9vUXM9V86WQxhwzTte3slDJqfo6QiJLUwh4ngfjNMr3q6 + dc9idSnFH4oZ08ZW9wuN7O5LT8vYhfF3Tppw / YP0aH5RoAs7momCCo7uIm + dEO8 = "
   }
   

2. Sample code to use AWS SDK to retrieve the file from amazon S3 bucket using temporary credentials from STS service:

   2.1 Call the Temporary Token Service to obtain the AccessKeyId, SecretAccessKey and SessionToken;

   
   

   2.2 Use the BasicSessionCredentials AWS SDK constructor to obtain temporary credentials object. Please, find here more details;

   BasicSessionCredentials basicSessionCredentials = new BasicSessionCredentials(accessKeyId, secretAccessKey, sessionToken); // See the mapping table below
   

   Mappings:

   Temporary Token service response	AWS SDK BasicSessionCredentials parameters
   awsCredentials.AccessKeyId	accessKeyId
   awsCredentials.SecretAccessKey	secretAccessKey
   awsCredentials.SessionToken	sessionToken

   
   

   2.3 Create the Amazon S3 Client using the temporary credentials obtained in step 2.2;

   AmazonS3 s3 = AmazonS3ClientBuilder.standard().withCredentials(new AWSStaticCredentialsProvider(basicSessionCredentials))
               .build();
   

   
   

   2.4 Use the GetObjectRequest AWS SDK API to generate the S3 object request to the access point provided in the Notification Response. Please, find here more details;

   GetObjectRequest getObjectRequest = new GetObjectRequest(accessPoint, key) // See the mapping table below
   

   Mappings:

   NotificationMessage	AWS SDK GetObjectRequest parameters
   MessageReference.objectKey	key
   MessageReference.accesspoint	accessPoint

   
   

   2.5 Use the S3 AWS SDK client to retrieve the file using the object request described in step 2.4;

   InputStream in = s3.getObject(getObjectRequest).getObjectContent();
   

   
   

Sample code to download a file:

BasicSessionCredentials basicSessionCredentials = new BasicSessionCredentials(accessKeyId, secretAccessKey, sessionToken);
AmazonS3 s3 = AmazonS3ClientBuilder.standard().withCredentials(new AWSStaticCredentialsProvider(basicSessionCredentials))
                .build();
GetObjectRequest getObjectRequest = new GetObjectRequest(accessPoint, key);
InputStream in = s3.getObject(getObjectRequest).getObjectContent();

Option 2: Using pre-signed URLs

Once EMR middleware receives the file availability notification (Step 2), file retrieve service (Step 3) can be called using the pre-signed URL received as parameter in:

• MessageReference from Step 2;

The file available for EMR consumption is available in Amazon S3 bucket using a pre-signed URL.

1. Notification message is received (Step n2);

   Sample Notification message:

   {
   "EmrOid": "2.16.724.4.2",
   "MessageId": "e6950ef2-3c66-4d00-9aba-205fde0d3200",
   "MessageType": "Clinical Report",
   "MessageCreatedAt": "1603778420307",
   "MessageReference": {
       "accesspoint": "https://{env}-emr-integration-data-store-s3.s3.amazonaws.com/delivery/2291405/cda/xxx.xml?X-Amz-Security-Token=yyy"
   },
   "MessageReceiptHandle": "abc"
   }
   

2. Sample code to retrieve the file from the s3 bucket using pre-signed URL:

   Mappings:

   NotificationMessage	Method to download file
   MessageReference.accesspoint	presignedUrl

   public byte[] getS3ObjectAsByteArrayByPresignedUrl(String presignedUrl) {
           URL presignedUrlData;
           ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
           InputStream inputStream = null;
           try {
               presignedUrlData = new URL(presignedUrl);
               URLConnection urlConnection = presignedUrlData.openConnection();
               logger.info("S3 Object content type:{} size:{} bytes", urlConnection.getContentType(), urlConnection.getContentLength());
               inputStream = presignedUrlData.openStream();
               byte[] byteChunk = new byte[EmrInternalMockConstants.BYTE_ARRAY_BUFFER_SIZE];
               int n;
               while ((n = inputStream.read(byteChunk)) > 0) {
                   byteArrayOutputStream.write(byteChunk, 0, n);
               }
           }
           return byteArrayOutputStream.toByteArray();
   ...