Quantum secure direct communication (QSDC) based on entanglement can directly transmit confidential information. However, the inability to simultaneously distinguish the four sets of encoded entangled states limits its practical application. Here, we explore a deterministic QSDC network based on time-energy entanglement and sum-frequency generation. 15 users are in a fully connected QSDC network, and the fidelity of the entangled state shared by any two users is greater than 97%. The results show that when any two users are performing QSDC over 40 kilometers of optical fiber, the fidelity of the entangled state shared by them is still greater than 95%, and the rate of information transmission can be maintained above 1Kbp/s. Our Letter demonstrates the feasibility of a proposed QSDC network, and hence lays the foundation for the realization of satellite-based long-distance and global QSDC in the future.