Showing posts with label research design. Show all posts
Showing posts with label research design. Show all posts

Saturday, July 10, 2021


Many patients read "sciency" sounding posts on the internet and youtube videos. They follow poor advice, thinking it is scientific. I compiled this checklist for the patient who is wondering if what he saw is science or fake news. Often, it looks like science because there are a lot of footnotes. I adapted a document I saw on Twitter.  I have added to it and added some explanation below:

Some Characteristics of Pseudoscience

1. Is UNFALSIFIABLE (can’t be proven wrong); makes vague or unfalsifiable claims.

2. Relies heavily on ANECDOTES, personal experiences, testimonials, “professional” opinions, and preclinical (test tube or animal) studies. IGNORES “LEVELS OF EVIDENCE,” and GRADE given by professional consensus.

Note: For every 10,000 compounds screened->250 (2.5%) are entered into a preclinical study 
-> 5 (2%) are tested in clinical trials -> 1 gets FDA approval

3. CHERRY PICKS confirming evidence while ignoring/minimizing disconfirming (especially higher level) evidence.

4. Uses TECHNOBABBLE: Words that sound scientific but don’t make sense.

5. Lacks PLAUSIBLE MECHANISM: No way to explain it based on existing knowledge, or deficient evidence for the proposed mechanism.

6. Is UNCHANGING: doesn’t self-correct or progress.

7. Makes EXTRAORDINARY/EXAGGERATED CLAIMS with insufficient clinical evidence.

8. Professes CERTAINTY; talks of “proof” with great confidence. Ignores statistical confidence intervals and power.

9. Commits LOGICAL FALLACIES: Arguments contain errors in reasoning.

10. Lacks PEER REVIEW: Goes directly to the public (e.g. YOUTUBE videos, blogs, direct-to-patient presentations only), avoiding scientific scrutiny.

11. Claims there is a CONSPIRACY (e.g., Big Pharma/FDA conspiracy) to suppress their ideas.

12. OVERSIMPLIFIES biochemistry (e.g. alkaline water, reducing sugar intake, antioxidants or anti-inflammatories will slow cancer)

13. Ignores INTERACTIONS with other substances, bioavailability, biochemical feedback effects, microbiome, substance purity, or adulteration

14. Claims “causation” when only “ASSOCIATION” has been demonstrated. (See the Bradford-Hill checklist) 

15. LACK OF DISCUSSION of potential biases, missing confounding variables, effects that may have changed over time and/or with improved technology.

16.  INAPPROPRIATE STATISTICS AND RESEARCH METHODS. Non-valid endpoints or subset conclusions, lack of pre-announced endpoint and subsets, lack of power to detect endpoint within sample size and timeframe, poor choice of surrogate endpoint or subsets, "p hacking," biases in retrospective studies.

17. Failure to disclose CONFLICTS OF INTEREST or sponsors.

#15 and #16 require some explanation:

Surrogate endpoints: Ideally, we would have long-term follow-up until death ("overall survival") for every trial. This is impractical, particularly for prostate cancer that has a very long natural history. ICECAP has identified "metastasis-free survival" as an appropriate surrogate for overall survival in trials involving men with localized prostate cancer.  The appearance of metastases has been suggested as appropriate for men with recurrent PCa but requires validation. Biochemical recurrence-free survival is only useful for predicting the success of a therapy for localized prostate cancer. PSA doubling time is definitely inappropriate without a control group. Radiographic progression-free survival seems to be a good surrogate endpoint in men who are metastatic and castration-resistant (see this link and this one). If the pattern holds, PSA-based endpoints are inadequate (see this link and this one) and only metastasis-based endpoints are adequate. Typically, trials are only powered (have enough sample size) to reliably detect differences in their primary endpoint.

Subset conclusions: Because there is only enough sample size to reliably detect differences in the primary endpoint, subset analysis is suspect. Using subset analysis, Spears et al. showed that men diagnosed on Mondays did not benefit from abiraterone - a ridiculous conclusion. They also showed that men diagnosed with metastases (M1) benefited while men diagnosed without metastases (M0) did not. Both conclusions are inappropriate. In the case of men without metastases, there were only 34 deaths among the 460 patients in the treatment group and 44 deaths among the 455 patients in the treatment group - not enough to prove a statistically significant effect with 95% confidence. However, with time, there may be enough deaths to achieve a statistically significant effect, so we have to be cautious about labeling it as ineffective in the M0 subgroup.

"P hacking" or "data-dredging"/positive results bias occurs when researchers do not announce before the study begins exactly which subgroups or variables will be looked at and which measures will be used to judge success or failure. They are going on a fishing expedition to find at least some variable or subgroup with statistically significant results. Because of random probabilities, if there are enough variables there will almost always be some that have statistically significant outcomes, like the "Monday diagnosis" subgroup above. Starting in 2000, all peer-reviewed journals required researchers to state upfront what they would be looking for. This made a large change in the number of positive results reported (see this link). Journals often would not print negative findings. In 2017, NIH and the FDA required the sponsors of all clinical trials listed in to provide results whether positive or negative. Policing and compliance are spotty.

Biases in retrospective studies and database analyses: Common biases are selection bias, ascertainment bias, lead-time bias, length bias, survivorship bias, confounding by unmeasured variables, and others.

Friday, July 10, 2020

The Perils and Pitfalls of "Treating PSA" in Advanced Prostate Cancer

Prostate Specific Antigen (PSA) is a protein on the surface of all benign prostate cells and most malignant prostate cancer cells. In prostate cancer, expression of PSA is correlated with the size of the tumor (see this link). When prostate cancer first metastasizes, the tumor is limited in size by its blood supply. As it grows, the cancer creates its own blood supply by secreting growth factors called VEGF. The PSA from the cancer activates VEGF to form blood vessels that bring oxygen and nutrients to the cancer and lymph vessels to drain fluids from the growing tumor (see this link). Tumor blood supplies are not as patent as those of benign tissues. Healthy prostate tissues with patent blood supply, and micrometastases that have little or no blood supply put out very little detectable PSA into the serum (although the cells express high levels of PSA). But the leaky blood supply of tumors allows PSA to enter the serum where it is detected by a PSA test. So, the larger, more established tumors of a given patient create almost all of his detectable PSA (see this link).

"Treating PSA"

I. Selecting for low PSA subtypes

For most men with advanced prostate cancer, PSA is their best biomarker of progression - more detected PSA means more progression. This may change as the cancer evolves. A highly mutated tumor may put out less PSA. Highly undifferentiated kinds of prostate cancer, and other relatively rare sub-types (e.g., ductal, neuroendocrine, basal cell, "double negative," etc.) may evince little or no serum PSA.  

So it is possible, when such phenotypes are present and they are mixed with "normal" prostate cancer, to provide treatments that kill off the "normal" prostate cancer cells, leaving the low-PSA subtypes behind. Such a situation has been identified in patients heavily treated with chemo and enzalutamide. It is called "treatment-emergent neuroendocrine prostate cancer" (see this link) and has been identified in 17% of heavily-treated patients. 

Another example of a treatment that may select for low-PSA subtypes is Lu-177-PSMA. If the patient has two types of prostate cancer, one that expresses PSMA and PSA, while his other cancer expresses neither, PSMA-targeted therapy may eliminate the source of most of the PSA, leaving more virulent subtypes behind (see this link). 

This type of situation is dangerous if one relies on PSA as the principal biomarker of progression. One may be lulled into complacency by deceptively low PSA.

It is worth noting that two FDA-approved therapies for prostate cancer, Provenge and Xofigo,  have been proven to increase survival, but have little or no effect on PSA.

II. Supplements that interfere with PSA tests

Patients often self-medicate in the hope of wresting some control over their cancer. The internet is full of "evidence" that this or that natural supplement may slow progression or even cure the cancer.  Serum PSA is detected by an antibody that can detect amounts as low as a nanogram of PSA per ml of serum. This kind of sensitivity has a cost - the antibodies are subject to interference by other substances that may be present in the serum. So far, the list of substances that may interfere with PSA tests, creating false negatives, includes biotin, curcumin, genistein, EGCG, resveratrol, capsaicin, saw palmetto, pygeum, beta-sitosterol, and statins (see this link). The false negative PSA readings may fool the patient and his physician (who may not be aware of the patient's supplement use) into believing that the cancer is under more control than it really is. Patients who use any complementary therapies are twice as likely to die of their cancer (see this link).

III. SBRT of oligometastases

1. Exponential growth

Because of Covid-19, many of us are now used to seeing exponential growth curves. Deaths from Covid-19 started very slowly in December through February. But then in March, the number of deaths climbed markedly. This illustrates the two striking features of exponential growth - the "flat" part with a very slow increase, followed by a "steep" part with a very rapid increase.

Among the biological systems that also follow an exponential growth curve are bacteria, viruses, and cancers. Here is a prototypical graph of the number of metastases in a patient.

In men who are PSA-recurrent after prostatectomy, it takes a median of 8 years for the first metastasis to become detectable (see this link). After that, I've seen that more than a year can go by between the detection of the first metastasis and the next one. Some researchers, who should know better, observed that in their patients who had early metastases treated with radiation, new metastases did not occur for a long time. They attributed the delay to the treatment rather than the natural history of metastatic progression  (see this link). It is impossible to know if there was a delay in progression without a randomized clinical trial.

What is really happening during this extended time period? The accepted theory is called "seed and soil." There are millions of cancer "seeds" in the serum, the lymph, around nerves, and hiding in various tissue reservoirs (mainly in bone tissue). While they appear to be quiescent, they are in fact changing the "microenvironment" of the tissue they are in. They are transforming the tissue to make it more conducive to prostate cancer growth, building networks of collagen, fat, blood vessels and nerves, influencing healthy cells to become cancerous, and preventing the immune system from destroying the new nests (see this link for a fuller explanation).

Because it takes such a long time to build up the metastases to the point that they are detectable by even our most sensitive PET/CT scan (the tumor detection limit is about 4 mm - millions of cells), it seems that there is little there and even less going on. This is called "oligometastatic" cancer. It seems like all the cancer can be picked off by playing whack-a-mole -- zapping the few detected metastases with intense radiation (called SBRT) as they are detected. In fact, it is well-established that SBRT provides excellent "local control." "Local control" means that the metastases are usually completely annihilated by just one or two "zaps" (see this link). Because the detected metastases are the source of almost all the PSA, PSA can fall to undetectable levels after such treatment of oligometastases. But the cancer is far from cured - the PSA has been treated, but the cancer is still micrometastatic and systemic.

Those who believe that such treatment can result in a durable remission believe that the immune system can clean up the rest of the cancer.  The ORIOLE trial (reviewed here) showed that SBRT created a T-cell response. If that T-cell response is sustained, they argue, the activated immune system can "clean up" the rest of the cancer. The skeptics argue that T-cell responses are usually not sustained. Trials of numerous immunotherapies (e.g., Prostvac, GVAX, GM-CSF, etc.) have failed to show a benefit because the early T-cell responses are countered by adaptive responses. Prostate cancer is notoriously "cold" to immunotherapies.

2. PSA-based Endpoints

What we really want to know is this: will the treatment enable patients to live longer? Overall survival is the gold standard of success of randomized clinical trials. The "problem" for clinical trials is that prostate cancer is such a slow killer, that it may take 15 years or more to discern a difference (see this link) if patients have localized or recurrent prostate cancer at the start. (For most other types of cancer, 5-year overall survival is more than adequate.) Clinical trials are often ended when half of the control group die (median survival). But, depending on patient characteristics at the start, median survival may never be reached within the duration of the clinical trial (see this link and this one and this one).

Prostate cancer-specific survival (how long before patients succumbed to their prostate cancer) is little better. It is also hampered by the fact that patients with prostate cancer may die of something else sooner, possibly because their cancer was debilitating. It is often unclear to the doctor who signs the death certificate whether the cancer was the end cause, a contributing cause, or a non-contributing factor. To get clinical trial results before new medical science and technology renders the results irrelevant, we want to use surrogate endpoints that are highly correlated with and predict overall survival.

The earliest endpoints that can be used to measure the success of a prostate cancer therapy are PSA based. All of the following surrogate/secondary endpoints are PSA based:
  • PSA50 - the percent who had a reduction in PSA by 50% or more
  • Nadir PSA - the lowest PSA reached after therapy (see this link)
  • PSA doubling time (PSADT) - whether the therapy slowed PSA growth
  • Biochemical recurrence (BCR) - depending on initial treatment, and there may be multiple salvage therapies, each with a PSA failure defined for it (see this link)
  • Biochemical Recurrence-Free Survival (bRFS)
  • Biochemical Disease-Free Survival (bDFS)
  • Biochemical failure (BF)- rise in PSA by a pre-specified amount post-therapy
  • Biochemical No Evidence of Disease (bNED)
  • Time to BCR/ BF
  • Time to start of lifelong ADT (based primarily on a pre-defined PSA failure benchmark)
  • Failure-free survival (FFS) or Progression-free survival (PFS) or Event-free survival (EFS) - defined as BF or radiological progression or clinical progression or death. 
The following surrogate endpoints are not PSA-based:
  • Clinical Progression-Free Survival (cPFS) - worsening of symptoms or performance status (see this link)
  • Radiographic Progression-free Survival (rPFS) or Disease-free survival (DFS)- progression on scans or death
  • Objective Response Rate (ORR) - tumor size or number reduction using RECIST criteria
  • Change in Bone Scan Index
  • Time to radiographic progression or failure
  • Metastasis-free survival
  • Clinical progression - pain, bone fracture, spinal compression
As an example of circular reasoning, we can see in the ORIOLE trial that 6-month Progression Free Survival (PFS) was chosen as the primary endpoint. PFS was defined as  PSA progression (by >25% over nadir and by > 2 ng/ml) or radiographic progression or death. As we can readily see in the exponential growth curve, the odds of a new metastasis on a bone scan/CT are very low and there are not likely to be any deaths. Therefore, PFS was almost entirely PSA progression. But the protocol "treated PSA." It is therefore illogical to conclude, even for a Phase II trial, that oligometastatic treatment slowed progression.

(Update 8/25/2022) Deek et al. combined ORIOLE and STOMP (n=162) with extended follow-up. After 52.5 months of median follow-up, they report:
  • Progression-free survival (PFS) was 11.9 mo. for metastasis-directed therapy (MDT) vs. 5.9 mos. for observation. (HR=0.44)
  • Radiographic progression-free survival (rPFS) was not significantly different
  • Time to castration resistance was not significantly different
  • Overall survival was not significantly different
  • PFS increased by about 5-6 months regardless of whether there were high-risk mutations (BRCA, ATM, RB1, TP53).
  • rPFS did not significantly increase for either group.

What is confusing is the endpoint used in this analysis. 

Progression-free survival (PFS) = 

  1. a PSA rise, or 
  2. radiographic progression, or 
  3. new symptoms, or 
  4. initiation of ADT, or 
  5. death.

In 52.5 months, there was very low mortality (5), and asymptomatic local control is good (3). Initiation of ADT (4) is always based on either rise in PSA (1) or radiographic progression (2). So with no difference in rPFS, the difference between PFS and rPFS is just PSA. This suggests that the extended follow-up found that MDT only treated PSA without any real impact on survival or progression of the cancer.

It is worth noting that radiation of the prostate ("debulking") has no survival or progression advantage when there are multiple metastases, only when the metastatic burden is low (see this link). The prostate is, of course, the source of all metastases, and an ideal environment for metastases to develop and grow. Metastasis-to-prostate spread has been observed. In a meta-analysis of the two debulking trials called STOPCAP M1, researchers found that there was a statistically significant reduction in PSA progression (by 26%), even when there was no benefit in terms of metastatic progression or survival. Treating PSA even by debulking the entire prostate is not in and of itself of any oncological benefit (there may be a palliative benefit, however).

3. Danger of Withholding Early ADT

While ORIOLE, STOMP, and SABR-COMET were Phase 2 clinical trials whose results were not meant to change practice, many patients and their doctors (often under pressure from patients) would like to believe they do. If the metastases are in places that are safe to irradiate (e.g., away from the mediastinum), there is little risk in doing so. However, if they do not understand the circular reasoning evident in the ORIOLE trial, they may put off therapies that are known to increase survival. There is also a risk of unreasonable expectations.

Some patients (and doctors) believe that by delaying ADT, they can increase their quality of life, and delay castration resistance. Neither is true. Contrary to popular belief, decreasing the intensity of hormone therapy and delaying its use brings earlier castration resistance and death. The strongest evidence for this comes from the STAMPEDE (on Zytiga and Xtandi), LATITUDE, and SPARTAN trials. Among men who were newly diagnosed with metastatic prostate cancer:
  • Overall survival was longer if men used Zytiga + ADT.
    • No difference based on the number of metastases
    • Failure-free survival was longer if they used Zytiga  + ADT
  • Overall survival was longer if men used Xtandi+ADT
    • Survival was especially lengthened if there were fewer metastases 
    • PSA progression-free survival was longer if they used Xtandi+ADT
  • Overall survival was longer if men used Erleada+ADT
    • PSA progression-free survival was longer if they used Erleada+ADT
A clear pattern emerges: early use of intensive hormone therapy prolongs survival and prolongs the time to castration resistance. Men who were oligometastatic benefited from early, intense hormone therapy.

The TROG 03.04 RADAR trial examined the duration of hormone therapy in high-risk men treated with radiation.  They found that, after 10 years of follow-up, men treated with 18 months of ADT survived longer, and reached castration resistance later compared to men treated with 6 months of ADT.

The TOAD trial looked at starting ADT at the first sign of recurrence vs. waiting for metastases to be detected. Men treated earlier reached castration resistance later. It also showed there was no major detriment to global health-related quality of life by starting ADT earlier (see this link).

Maha Hussain reported the results of a randomized clinical trial comparing intermittent vs continuous ADT in recurrent men with metastases. She found that:
  • Time to castration resistance was not different for the two protocols (Figure S5)
  • For men with minimal disease, overall survival was 6.9 years for those on continuous therapy vs 5.4 years for those on intermittent therapy. The trial was underpowered for this difference to reach statistical significance.
  • It took 4-5 years for the survival curves to start separating - long follow-up is needed to detect survival differences.
Taken together, all these major randomized clinical trials show that the best way to use ADT in the oligometastatic setting is to use it early and heavily. Reducing the number of cancer cells as quickly and effectively as possible, even reducing those cells that haven't begun to measurably contribute to PSA, extends survival. The effect of evolutionary selection pressure allowing castration-resistant cells to survive is dwarfed by the reduction in sheer numbers. Circular reasoning may harm patients.

4. Future Clinical trials

We have learned some lessons about clinical trials for oligometastatic treatment:
  • It has to have long enough follow-up, depending on the setting: at least 5 years for  newly diagnosed or recurrent men to allow time to get to the steep part of the exponential curve. It will take longer if more sensitive imaging is used.
  • It must use radiographic progression-free survival, or similar, as its primary endpoint
  • It must not use a PSA-related endpoint
  • ADT must be used in at least the control group. It would be unethical to withhold the standard of care (see AUA Guidelines for Advanced Prostate Cancer (mHSPC 14-18)) .
  • It should preferably use a PSMA PET/CT to locate metastases. The ORIOLE trial only found an advantage if patients were oligometastatic on both a PSMA PET/CT and a bone scan/CT. The use of more sensitive imaging will move the starting point to the left on the exponential curve, so it will take that much longer to detect a benefit.
These randomized clinical trials (RCTs) are currently active:
  • The CORE RCT (active, no longer recruiting) at Royal Marsden Hospital in London will have 5 years of follow-up (completion in 2024) and will include freedom from widespread metastatic disease and overall survival among the outcomes looked at. 
  • The PCX IX RCT (among castration-resistant patients) at Jewish General Hospital in Montreal will have 5 years of follow-up (primary outcome in 2025) and has radiographic progression-free survival as its primary outcome. 
  • The PLATON RCT (among hormone-sensitive patients) in Canada will have 6 years of follow-up (primary outcome in 2025) and has radiographic progression-free survival as its secondary outcome. Oligometastatic men who have never had their prostates treated with RT will have prostate radiation too in both arms. ADT is given in both arms, advanced hormonals and chemo at the physician's discretion.
  • The STEREO-OS RCT (recruiting, study completion in 2022) in France will look at radiographic progression-free survival with follow-up of up to 3 years. 
  • The FORCE RCT at the University of Michigan (primary completion in 2022) will compare systemic treatment with ADT and any of Taxotere, Zytiga or Xtandi (at the discretion of the treating physician) to similar systemic treatment plus metastasis-directed SBRT for men with mCRPC who have not yet had any of those advanced systemic therapies. They will evaluate progression-free survival after 18 months. "Progression" is defined as alive and at least a 20% increase (and at least 5 mm net increase) in the size of tumors or any new metastases. They will detect metastases via bone scan/CT, However, they will also test whether PSMA-based PET indicators are as useful among men with mCRPC as it is in men with newly recurrent disease.
  • The VA STARPORT RCT (primary completion in 2025) in many VA hospitals in the US will randomize patients to systemic therapy + PET-directed radiation to 1-5 oligorecurrences or to systemic therapy alone. Unfortunately, they are using castration-resistance as their primary endpoint, which is problematic.
  • The START-MET RCT (primary completion in 2025) in Spain will randomize recurrent and newly diagnosed oligometastatic (≤3 on bone scan/CT and ≤5 on PSMA PET) men to standard-of-care (ADT+2nd line HT+prostate RT) or standard-of-care + SBRT to all metastases. 2-year radiographic progression is the primary outcome.
  • The SPARKLE RCT (primary completion in 2027) in Belgium randomizes oligo-recurrent patients to either (1) MDT alone, (2) MDT+1 mo.of ADT or (3)MDT+6 mo (ADT+enzalutamide). Primary endpoint is 5 new lesions on PSMA PET scan.
  • The ADOPT RCT (primary completion in 2022) in The Netherlands randomizes oligo-recurrent patients to either MDT ± ADT. 2.5 yr MFS on PSMA PET scan.