Lab Results Upload and Parsing

To analyze a lab report file (PDF or image file), you first need to upload the file to SD's Amazon S3 bucket.

1. Get presigned URL to upload lab report file

To upload to SD's Amazon S3 bucket, you will need a URL to send a POST request to containing the file, and the fields that you are supposed to send along with the file for verification purposes.

To get the URL and the relevant fields, call the GET /ocr/get_presigned_url/ API.

As shown, the API will return the url and the fields for authentication purposes. This is what a sample response looks like:

{
  "url": "https://sd-platform-qa-userdatas3bucket-v4i0in6l6luu.s3.amazonaws.com/",
  "fields": {
    "key": "tmp/ocr/requests/0423691b-ccab-442b-ab3e-7948448c0098.pdf",
    "AWSAccessKeyId": "ASIA24LREHUCUF5WBINM",
    "x-amz-security-token": "IQoJb3JpZ2luX2VjEF0aCXVzLWVhc3QtMSJFMEMCH37izMoJJmUZ1qr+sO93nBQaAfyDfTEhmdJmCJhXv48CICx3P2zxJ8rsu3+a/rrJCd7Y6NxvjeujyuC8ubVz76VtKq4CCDYQAxoMNzQ4MDk4NDM2MzU3IgzIZQklcSllNA+3PGwqiwKEnluuiCAPnCnc/BedKNBFGfBO1dS8vjGrMe5Yv+d21exAmBuPGKULO37gVgPb6l/nJunljZGNhXGoHISc3l1GS/r2gAMHjXONWl7+mRNzjVeXRYz+NIVGdgbu5pQzz4NY6q066MxXURPeMTEt5y6Mhgdcqhaz8MMqSvG3xTAVmIKLkTih5wYEKfPBylxTM+ugM8UaQq2+aUYgQmZ7e3+mqpSrkP/HEYDEIkGjuLErNF7TYOwIgbErPOBdw/pGjd4wyqSOVOYxGuKzbu4cgPJExoa3slZ3H1WbzvzM5qDEx88kGe/KriFNHS/Q9Vn9ueVRqUOZGhSUrS9bpJgXjD8jVCpS1ndwArU8M2Ewo8HwkwY6nAFFPkATmSZHlfNfJmq8D3mn0lSCsrVkQh/928FLDWqMnzvYxfMDNnV8/03JVIr9qCwts6m69fU5oIeTE1/d8s0EiPpl8GSs61cWI1/hOKTEMSGetFQECxYNJYVIChqJd9vGV8A8yMt0p7hwSXl5yCnYR+hDwigejOWY2jumUVpzItg8KjwutcvO3x0OC/8SSKQ7ELGIBt15+6qLBD0=",
    "policy": "eyJleHBpcmF0aW9uIjogIjIwMjItMDUtMTFUMjA6NTE6MzVaIiwgImNvbmRpdGlvbnMiOiBbeyJidWNrZXQiOiAic2QtcGxhdGZvcm0tcWEtdXNlcmRhdGFzM2J1Y2tldC12NGkwaW42bDZsdXUifSwgeyJrZXkiOiAidG1wL29jci9yZXF1ZXN0cy8wNDIzNjkxYi1jY2FiLTQ0MmItYWIzZS03OTQ4NDQ4YzAwOTgucGRmIn0sIHsieC1hbXotc2VjdXJpdHktdG9rZW4iOiAiSVFvSmIzSnBaMmx1WDJWakVGMGFDWFZ6TFdWaGMzUXRNU0pGTUVNQ0gzN2l6TW9KSm1VWjFxcitzTzkzbkJRYUFmeURmVEVobWRKbUNKaFh2NDhDSUN4M1AyenhKOHJzdTMrYS9yckpDZDdZNk54dmpldWp5dUM4dWJWejc2VnRLcTRDQ0RZUUF4b01OelE0TURrNE5ETTJNelUzSWd6SVpRa2xjU2xsTkErM1BHd3Fpd0tFbmx1dWlDQVBuQ25jL0JlZEtOQkZHZkJPMWRTOHZqR3JNZTVZditkMjFleEFtQnVQR0tVTE8zN2dWZ1BiNmwvbkp1bmxqWkdOaFhHb0hJU2MzbDFHUy9yMmdBTUhqWE9OV2w3K21STnpqVmVYUll6K05JVkdkZ2J1NXBReno0Tlk2cTA2Nk14WFVSUGVNVEV0NXk2TWhnZGNxaGF6OE1NcVN2RzN4VEFWbUlLTGtUaWg1d1lFS2ZQQnlseFRNK3VnTThVYVFxMithVVlnUW1aN2UzK21xcFNya1AvSEVZREVJa0dqdUxFck5GN1RZT3dJZ2JFclBPQmR3L3BHamQ0d3lxU09WT1l4R3VLemJ1NGNnUEpFeG9hM3NsWjNIMVdienZ6TTVxREV4ODhrR2UvS3JpRk5IUy9ROVZuOXVlVlJxVU9aR2hTVXJTOWJwSmdYakQ4alZDcFMxbmR3QXJVOE0yRXdvOEh3a3dZNm5BRkZQa0FUbVNaSGxmTmZKbXE4RDNtbjBsU0NzclZrUWgvOTI4RkxEV3FNbnp2WXhmTUROblY4LzAzSlZJcjlxQ3d0czZtNjlmVTVvSWVURTEvZDhzMEVpUHBsOEdTczYxY1dJMS9oT0tURU1TR2V0RlFFQ3hZTkpZVklDaHFKZDl2R1Y4QTh5TXQwcDdod1NYbDV5Q25ZUitoRHdpZ2VqT1dZMmp1bVVWcHpJdGc4S2p3dXRjdk8zeDBPQy84U1NLUTdFTEdJQnQxNSs2cUxCRDA9In1dfQ==",
    "signature": "Qvh/EA857VWcaiQKI87bt3TJ7A8="
  }
}

2. POST the file to the presigned URL

Make a POST request to the URL returned from Step 1 using the fields returned from Step 1 as payload, along with the lab results file that you wish to upload. A sample request would look like the following:

POST https://sd-platform-qa-userdatas3bucket-v4i0in6l6luu.s3.amazonaws.com/

key: tmp/ocr/requests/0423691b-ccab-442b-ab3e-7948448c0098.pdf
AWSAccessKeyId: ASIA24LREHUCUF5WBINM
x-amz-security-token: IQoJb3JpZ2luX2VjEF0aCXVzLWVhc3QtMSJFMEMCH37izMoJJmUZ1qr+sO93nBQaAfyDfTEhmdJmCJhXv48CICx3P2zxJ8rsu3+a/rrJCd7Y6NxvjeujyuC8ubVz76VtKq4CCDYQAxoMNzQ4MDk4NDM2MzU3IgzIZQklcSllNA+3PGwqiwKEnluuiCAPnCnc/BedKNBFGfBO1dS8vjGrMe5Yv+d21exAmBuPGKULO37gVgPb6l/nJunljZGNhXGoHISc3l1GS/r2gAMHjXONWl7+mRNzjVeXRYz+NIVGdgbu5pQzz4NY6q066MxXURPeMTEt5y6Mhgdcqhaz8MMqSvG3xTAVmIKLkTih5wYEKfPBylxTM+ugM8UaQq2+aUYgQmZ7e3+mqpSrkP/HEYDEIkGjuLErNF7TYOwIgbErPOBdw/pGjd4wyqSOVOYxGuKzbu4cgPJExoa3slZ3H1WbzvzM5qDEx88kGe/KriFNHS/Q9Vn9ueVRqUOZGhSUrS9bpJgXjD8jVCpS1ndwArU8M2Ewo8HwkwY6nAFFPkATmSZHlfNfJmq8D3mn0lSCsrVkQh/928FLDWqMnzvYxfMDNnV8/03JVIr9qCwts6m69fU5oIeTE1/d8s0EiPpl8GSs61cWI1/hOKTEMSGetFQECxYNJYVIChqJd9vGV8A8yMt0p7hwSXl5yCnYR+hDwigejOWY2jumUVpzItg8KjwutcvO3x0OC/8SSKQ7ELGIBt15+6qLBD0=
policy: eyJleHBpcmF0aW9uIjogIjIwMjItMDUtMTFUMjA6NTE6MzVaIiwgImNvbmRpdGlvbnMiOiBbeyJidWNrZXQiOiAic2QtcGxhdGZvcm0tcWEtdXNlcmRhdGFzM2J1Y2tldC12NGkwaW42bDZsdXUifSwgeyJrZXkiOiAidG1wL29jci9yZXF1ZXN0cy8wNDIzNjkxYi1jY2FiLTQ0MmItYWIzZS03OTQ4NDQ4YzAwOTgucGRmIn0sIHsieC1hbXotc2VjdXJpdHktdG9rZW4iOiAiSVFvSmIzSnBaMmx1WDJWakVGMGFDWFZ6TFdWaGMzUXRNU0pGTUVNQ0gzN2l6TW9KSm1VWjFxcitzTzkzbkJRYUFmeURmVEVobWRKbUNKaFh2NDhDSUN4M1AyenhKOHJzdTMrYS9yckpDZDdZNk54dmpldWp5dUM4dWJWejc2VnRLcTRDQ0RZUUF4b01OelE0TURrNE5ETTJNelUzSWd6SVpRa2xjU2xsTkErM1BHd3Fpd0tFbmx1dWlDQVBuQ25jL0JlZEtOQkZHZkJPMWRTOHZqR3JNZTVZditkMjFleEFtQnVQR0tVTE8zN2dWZ1BiNmwvbkp1bmxqWkdOaFhHb0hJU2MzbDFHUy9yMmdBTUhqWE9OV2w3K21STnpqVmVYUll6K05JVkdkZ2J1NXBReno0Tlk2cTA2Nk14WFVSUGVNVEV0NXk2TWhnZGNxaGF6OE1NcVN2RzN4VEFWbUlLTGtUaWg1d1lFS2ZQQnlseFRNK3VnTThVYVFxMithVVlnUW1aN2UzK21xcFNya1AvSEVZREVJa0dqdUxFck5GN1RZT3dJZ2JFclBPQmR3L3BHamQ0d3lxU09WT1l4R3VLemJ1NGNnUEpFeG9hM3NsWjNIMVdienZ6TTVxREV4ODhrR2UvS3JpRk5IUy9ROVZuOXVlVlJxVU9aR2hTVXJTOWJwSmdYakQ4alZDcFMxbmR3QXJVOE0yRXdvOEh3a3dZNm5BRkZQa0FUbVNaSGxmTmZKbXE4RDNtbjBsU0NzclZrUWgvOTI4RkxEV3FNbnp2WXhmTUROblY4LzAzSlZJcjlxQ3d0czZtNjlmVTVvSWVURTEvZDhzMEVpUHBsOEdTczYxY1dJMS9oT0tURU1TR2V0RlFFQ3hZTkpZVklDaHFKZDl2R1Y4QTh5TXQwcDdod1NYbDV5Q25ZUitoRHdpZ2VqT1dZMmp1bVVWcHpJdGc4S2p3dXRjdk8zeDBPQy84U1NLUTdFTEdJQnQxNSs2cUxCRDA9In1dfQ==
signature: Qvh/EA857VWcaiQKI87bt3TJ7A8=
file: (binary)

You'll get a "Status Code: 204 No Content" as the response

3. Initiate Object Character Recognition (ocr) of the recently uploaded file

Now that the file has been uploaded to s3, it has to be read and the lab marker names, units, values and date of test needs to be extracted from the file. To proceed, call the POST /ocr/ endpoint endpoint with the two data fields: content-type and file.

Please note that the value of content-type data field in the payload would depend on the type of file. For example, if it is a PDF file, which would be the case most of the times, the content-type field should be set to "application/pdf" or if it is a PNG image file, the content-type would be "image/png" instead.

The file field would contain the key received from Step 1. It indicates the location of the uploaded file on Amazon S3 bucket.

Sample payload is given below.

{
  "content-type": "application/pdf",
  "file": "tmp/ocr/requests/0423691b-ccab-442b-ab3e-7948448c0098.pdf"
}

If the above request succeeds, request_id will be returned in the success response. You will then use the received request_id to query the status of the file. Sample response:

{
  "request_id": "0ac5bff7-6b21-4261-9ab3-85893e73c456"
}

4. Query the status of the file

Call the GET /ocr/ endpoint, with the request_id received from Step 3 as a query parameter. If you make the request while the file is being read and processed, you'll receive the response:

{
  "status": "pending",
  "message" "No response found for given request_id, please try again after sometime"
}

Once the file has been processed, you'll receive the success response. Keep calling the API until the success response is received (which would take some seconds). Sample response is given below:

{
  "request_id": "0ac5bff7-6b21-4261-9ab3-85893e73c456",
  "status": "successful",
  "description": "Success.",
  "date": "2021-08-10",
  "results": [
    {
      "marker": {
        "id": "48f165d5-7b00-c7f4-781e-f86f5c8cc1ab",
        "name": "White Blood Cell Count",
      },
      "unit": {
        "id": "02f16d3d-8f4c-29e7-d92b-0703f7467ac9",
        "name": "Thousand/uL",
      },
      "level": 10,
      "discrete_value": null,
    },
    {
      "marker": {
        "id": "993955be-5888-6f39-137c-56af8c5187c1",
        "name": "Red Blood Cell Count",
      },
      "unit": {
        "id": "8d62d777-8090-c4bd-d6a7-4dbe3e572e0f",
        "name": "million/uL",
      },
      "level": 5,
      "discrete_value": null,
    },
  ],
}

The fields of major concern in the response are the date and results fields. The date field contains the date of the lab test, and the results field is an array with each item representing the information of each marker, its unit and its value read from the file. The level field represents the value of the numeric markers. While if there is a discrete marker, its value is populated in the discrete_value field.

Please note that the markers that are read from the file are matched to a list of supported markers stored in our database, and the matched markers are returned. So there is a possibility that there are some lab markers present in a lab results file, but not supported by SD, hence they would not be returned in the response.

5. Limitations of OCR and filling in missing data

The OCR is not always 100% accurate, and sometimes it may not correctly read each and every detail of each and every marker in the report. Sometimes, it may not read the date, or it may miss the marker unit or the marker value. In those cases, the JSON response from Step 4 may have some missing values. The values can be manually filled after making comparisons with the lab results file.

6. Getting details of the matched markers

Now that you have a list of matched lab markers returned in the results field from Step 4, you can fetch the details of those markers (like description of marker, the areas of the body it affects, its synonyms, list of units etc.).

To fetch details, call the /marker/ list endpoint, providing the marker IDs separated by comma, received from Step 4 as the query parameter.

7. Next steps

Now that the results have been parsed and read by the OCR, they can be analyzed. Analysis would give a clear picture of each marker and its value matched by the OCR, i.e. whether the value lies in optimal, sub-optimal or other range and what are the recommendations (if any) that can be helpful given the lab result. For details regarding analysis of the markers, please refer to the next page.