According to Illinois Poison Control, they are not poisonous. However, there is always the possibility of choking of ingested. The sap in the leaves of Ficus is toxic and can cause gastrointestinal problems.
While the seed can be used to start new plants, the germination is quite slow (up to 3 months). It's easier purchase an established one.