Neuroendocrine neoplasms (NENs) are a heterogeneous group of tumors with different immunophenotypes, morphological features, clinical presentations, and outcomes. Therefore, the diversity of NENs can easily confuse clinicians and lead to misdiagnosis. This article reports a case of a rectal NEN (r-NEN) growing within a lipoma with ulcerative lesions, which had been mistaken for rectal cancer. Fortunately, an experienced surgeon questioned the initial diagnosis (rectal cancer) after combining the results of a colonoscopic biopsy and the digital anorectal examination, thus enabling the patient to avoid unnecessary surgery. Through the review of this case and relevant literature, we further analyze the growth pattern of NENs and lipomas. Additionally, we hope to enhance clinicians’ awareness of a NEN that grew insidiously within a lipoma with surface ulcerative lesions, to make accurate diagnoses and treatments in time.