It depends on the cluster size how big the files can be. Take a look here. in the referenced support article it is described as follows:
All file systems that are used by Windows organize your hard disk based on cluster size (also known as allocation unit size ). Cluster size represents the smallest amount of disk space that can be used to hold a file. When file sizes do not come out to an even multiple of the cluster size, additional space must be used to hold the file (up to the next multiple of the cluster size). On the typical hard disk partition, the average amount of space that is lost in this manner can be calculated by using the equation (cluster size)/2 * (number of files).
If no cluster size is specified when you format a partition, defaults are selected based on the size of the partition. These defaults are selected to reduce the space that is lost and to reduce the fragmentation that occurs on the partition.
But it is also the maximum file size which depends on the cluster size and also on the sector size selected, because the file size is calculated as
File size = (number of clusters) * (sectors per cluster) * (sector size)
and the sector size is usually 512 bytes. but nowadays where we have hard disks with several terabytes capacity, it can be larger (e.g. 4 KB ).
While it seems that this allows larger files, the design specification of FAT32 says that a file cannot exceed 4 GB, hence
Max File size = 4 294 967 296 bytes (4 Gigabytes )
The maximum number of clusters for Fat32 is limited to 268 435 445. and a cluster must not exceed 64 KB, see here. But it is also said that you can't format Fat32 with larger clusters than 32 KB. This means that theoretically, the maximum partition size is
Max part. size = 268 435 445 * 32 768 bytes = 8 796 092 661 760 bytes (which is 8 Terabytes ).
Practically, it is lower than that, because usually you don't have 32 KB clusters. For Fat32 the default cluster size varies depending on the size of the partition between 4 KB (236 MB - 8 GB partitions) and 16 KB (16 GB - 32 GB partitions).
You can influence the maximum partition size when you are formatting the partition. The larger the cluster size and/or sector size, the larger the partition can be. But the file size is always limited to 4 GB.
It is not dependent on the medium, you can format a memory stick or SD card with larger sectors as you can do it with your harddisk. The limitation you have depends on the system which reads the partition, for example if you're using a digital camera it might not support each sector or cluster size as your operating system on the PC does.
Despite the fact that Microsoft has stopped support for Fat32 partitions larger than 32 GB, you can still create and use them, if you use a free 3rd party tool to format them. Fat32 is still used by many devices and in many operating systems (not just by Windows) such as HDD recorders, cameras etc.