Inclusion Criteria

• Has a histologically and/or cytologically confirmed diagnosis of NSCLC (squamous or nonsquamous)
• For Phase 2 of the study has a diagnosis of mixed squamous and nonsquamous (or adenosquamous) NSCLC
• Has locally advanced or metastatic disease (Stage IIIb or Stage IV) with radiographically or clinically evaluable lesions
• Has experienced failure of at least 1 prior chemotherapy regimen
  • For Phase 2 of the study
    • Participants must have received 1 prior platinum-based chemotherapy regimen (excluding a docetaxel-containing regimen) for advanced or metastatic (Stage IIIb or Stage IV) disease followed by documented PD
    • A drug provided as maintenance therapy following cytotoxic chemotherapy will be considered to be part of that regimen
    • Participants who received prior therapy with paclitaxel as a part of the platinum-based doublet front-line regimen without PD on therapy
    • Participants who, after the front-line, platinum-based, non-docetaxel containing chemotherapy, have been treated with 1 line of nivolumab or other immune-checkpoint inhibitors but progressed on or after the therapy
  • For Phase 1b of the study
    • Participants who have experienced failure of multiple lines of prior chemotherapy are eligible
• For Phase 2, has archived or fresh tumor biopsy samples (obtained during screening) sufficient for genotyping
• Has adequate organ function, before the first dose of study drug
• Has Eastern Cooperative Oncology Group (ECOG) Performance Status of 0 or 1
• Female participants who are postmenopausal for at least 1 year before the screening visit or are surgically sterile
• Are of childbearing potential and agree to practice 1 highly effective method and 1 additional effective (barrier) method of contraception, at the same time, from the time of signing the informed consent through 30 days (or longer, as mandated by local labeling) after the last dose of study drug, or agree to practice true abstinence
• Female participants must agree to not donate eggs (ova) during the course of this study and for 30 days after receiving their last dose of MLN1117 and, for docetaxel, for as long as is mandated by local labeling
• Male participants agree to practice highly effective barrier contraception during the entire study treatment period and through 120 days after the last dose of MLN1117 and, for docetaxel, for as long as is mandated by local labeling, or agree to practice true abstinence
• Male participants must agree to not donate sperm during the course of the study and for 120 days after receiving their last dose of MLN1117 and, for docetaxel, for as long as is mandated by local labeling
• Has suitable venous access for the study-required blood sampling
• Has recovered (ie, ≤ Grade 1 toxicity or eligibility per this protocol is met) from the reversible effects of prior anticancer therapy
• In the opinion of the investigator, the patient or legal guardian is capable of understanding and complying with protocol requirements for the duration of the study

Exclusion Criteria

• Previous treatment with a PI3K or AKT inhibitor
• Prior cancer therapy or other investigational therapy within 2 weeks before the first administration of study drug or failed to recover from the reversible effects of prior anticancer therapies
  • For prior therapies with a half-life longer than 3 days, the interval must be at least 28 days before the first administration of study drug, and the patient must have documented progressive disease
• Has poorly controlled diabetes mellitus defined as HbA1c > 6.5%
• Has taken strong inhibitors or strong inducers of CYP3A4 within 14 days before the first dose of study drug
• Has taken histamine-H2 receptor antagonists and/or neutralizing antacids within 24 hours before the first administration of study drug
• Has taken proton pump inhibitors within 7 days before the first administration of study drug
• Has a condition that requires the concomitant use of any of the protocol-excluded medications, supplements, or food products during the course of the study
• Has any clinically significant co-morbidities
• Has acute myocardial infarction within 6 months before starting study drug, current or history of New York Heart Association Class III or IV heart failure
  • Evidence of current uncontrolled cardiovascular conditions including cardiac arrhythmias, angina, pulmonary hypertension, or ECG evidence of acute ischemia or active conduction system abnormalities
  • Fridericia's corrected QT interval > 475 milliseconds (msec) (males) or > 450 msec (females) on a 12-lead ECG during the screening period
  • Abnormalities on 12-lead ECG including, but not limited to, changes in rhythm and intervals that in the opinion of the investigator are considered to be clinically significant
• Has known, previously diagnosed human immunodeficiency virus infection or active chronic hepatitis B or C
• Has brain metastasis, unless has completed definitive therapy, is not on steroids, has a stable neurologic status for at least 2 weeks after completion of the definitive therapy and steroids, and does not have neurologic dysfunction that would confound the evaluation of neurologic and other adverse events
• Has active secondary malignancy that requires treatment
• Has any serious medical or psychiatric illness, including drug or alcohol abuse
• Male participants who intend to donate sperm during the course of this study or 120 days after receiving their last dose of MLN1117 and, for docetaxel, for as long as is mandated by local labeling
• Female participants who are lactating and breastfeeding or have a positive serum pregnancy test during the screening period or a positive urine pregnancy test on day 1 before administration of the first dose of study drug
• Is unwilling or unable to abide by the requirements of the study